A person tells a cat from a human at a glance, but what a computer receives is a table packed with numbers from 0 to 255. We follow how a photo becomes a matrix down to a single pixel, see why pixel distance can’t measure “resemblance,” and then compute by hand a convolution that finds edges with a 3 × 3 filter. Finally, we look at the strengths and limits of hand-designed features (face boxes, landmarks, edge strength).
Pointy ears and fur versus smooth skin. A person tells them apart at a glance. What does a computer receive this photo as, and which clues can it find first?
Image created by the author. The lines and dots are facial landmarks defined in advance by a person.
The red lines on the cat at left are a facial skeleton connecting the forehead, eyes, and nose. The green box on the person at right is a detection result saying “there is a face here,” the red dots are landmarks along the jawline, eyebrows, and lips, and the pink dots mark the key positions of the two eyes, nose, and mouth. All of these are clues for which a person decided in advance “what to look at.”
This episode looks at two things. First, what a computer receives is not lines and dots like these but a matrix of numbers (Sections 2–3). Second, filters that find clues like edges in those numbers, and the limits of clues chosen by people (Sections 4–6).
For the number examples, the whole series follows the same single chihuahua photo (the star of 4-4). That’s because pixels and filters work the same way on any photo. In the hands-on activities, you can choose any of the five photos: cat, human, dog, chihuahua, and muffin.
Explained for Beginners
A problem that is easy for people isn’t necessarily easy for a computer. A person notices “it has fur” and “its ears are pointy” the moment they see a photo, but a computer has to compute each of those clues from numbers, one by one. In cat vs human, those clues are fairly clear, which makes it a good first exercise in image classification.
2Pixels
Pixels and Matrices — The Smallest Information a Computer Receives
A photo is a grid of numbers, and a color photo is three such grids stacked together. The smallest unit a model takes as input is a single channel value (0–255).
1Definition
A digital photo is a collection of tiny cells, finely divided across and down. Each of these cells is called a pixel (picture element). A pixel holds just one color, and that color is written as three intensities of red (R), green (G), and blue (B) light. Each intensity is an integer from 0 (no light) to 255 (brightest). So a color photo becomes a table of height × width × 3 numbers, that is, a matrix (strictly speaking, a three-layer tensor).
ShapeHeight H × width W × 3 channels
One valueInteger from 0 to 255 (8 bits)
Smallest unit of meaningOne channel value
Put simplyIt’s like coloring in graph paper. If you write down, from 0 to 255, how many drops from each of three paint pots (red, green, blue) went into each square, anyone can repaint the same picture from that note alone. What the computer has is not the picture but this note.
2Formula
X[y, x, c] ∈ {0, 1, …, 255}, y = 1…H, x = 1…W, c ∈ {R, G, B}Note: these 0–255 values are corrected (gamma-encoded) for display. Twice the number does not mean exactly twice the physical light intensity.
Grayscale luminance Y = 0.299·R + 0.587·G + 0.114·BA weighted sum reflecting that the human eye is most sensitive to green (ITU-R BT.601). One color cell shrinks to a single number.
Before feeding the model: x′ = x / 255 (to between 0 and 1)
H, W
Number of cells down and across. Together they are called the resolution
c (channel)
Color layer. Color has three (R, G, B); grayscale has one
X[y, x, c]
One number at row y, cell x, channel c. The smallest input the model receives
Y
Grayscale luminance made by mixing the three channels
Reading the Formula Piece by Piece
{0, 1, …, 255}This means there are 256 possible integer values. Since 2⁸ = 256, one number fits exactly in 8 bits (1 byte).
H × W × 3The count of numbers. The hands-on photos on this page have 56 × 56 × 3 = 9,408; a common model input of 224 × 224 × 3 has 150,528; a 12-megapixel phone photo has about 36 million.
0.299R + 0.587G + 0.114BThis has the same shape as a perceptron’s weighted sum. Multiply the three numbers by their shares and add them, and you get one brightness. The three shares add up to 1, so the result also stays within 0–255.
x / 255Scales large numbers down to 0–1 so the model can handle them more easily. It serves the same purpose as the normalization in the previous episode.
Try It with Numbers (the chihuahua photo reduced to 14 × 14)Eye cell (R, G, B) = (39, 37, 43) → Y = 0.299×39 + 0.587×37 + 0.114×43 = 38.3 ≈ 38 (nearly black) Fur cell (R, G, B) = (212, 161, 115) → Y = 0.299×212 + 0.587×161 + 0.114×115 = 171.0 ≈ 171 (light beige) The difference between the two cells is 171 − 38 = 133. This sharp difference is what gets picked up as an “edge” in Section 4.
3Diagram · Breaking It Down to the Smallest Information
4Interpretation
Keep breaking a photo down and you reach one pixel, the R, G, B numbers inside it, and the eight bits that make up one number. A bit is the smallest unit a computer stores, but a single bit is neither color nor brightness, just on or off.
So in image deep learning, the smallest piece of information with meaning is a single channel value: a number like “the red intensity of this cell is 39.” The model’s first layer takes exactly these numbers (X[y, x, c]) as input.
But this single number carries no meaning of “chihuahua” at all. An eye pixel (39, 37, 43) and a blueberry pixel (37, 31, 37) are almost the same as numbers. Meaning comes from how many pixels are grouped together, and finding those groupings is the feature extraction of Section 4.
5Animated Diagram · How a Photo Becomes Numbers
This shows the chihuahua photo cut into 14 cells across and down, with one eye cell picked and turned into numbers. The numbers on the right are computed from the photo’s actual pixels.
Original photo
Cut into a grid
Pick one cell
Three R·G·B numbers
One grayscale number
Spread out as a matrix
6Pixel Magnifier · Click to See the Numbers
Choose a photo and a resolution, then click a cell in the picture or hover over it. The table beside it (below it on mobile) shows the actual numbers of the chosen cell and its surrounding 7 × 7 cells. After clicking the picture, you can move between cells with the arrow keys.
Photo
Resolution
View
7Key Features · Using Raw Pixels
Strengths
The most honest input, with none of the photo’s information left out
Every photo has the same shape (H × W × 3), so they can all go into one model the same way
They’re numbers, so you can compute with them right away using multiplication and addition
Limitations
Too many numbers: 150,528 in a single 224 × 224 photo
Shifting a photo by just a cell or two, or brightening it a little, changes almost every number
A single number has no meaning. An eye and a blueberry can have the same numbers (covered in Section 4 and in 4-3 and 4-4)
Where it fits: before deep learning, people hand-crafted rules for extracting features from pixels (the landmarks in 04); since 2012, CNNs take raw pixels and learn the features on their own.
✓Check Question
Q1. How many numbers are in a color photo that is 10 cells wide and 10 cells tall?
Q2. What color is the pixel value (R, G, B) = (0, 0, 0)?
Q3. If you shift the same photo two cells to the right, what happens to the numbers at the same positions?
Q4. What is the smallest piece of information an image deep learning model takes as input?
Explained for Beginners
A pixel is a small cell that makes up a photo, and each cell records the intensities of red, green, and blue light as numbers from 0 to 255. Compared with the table in 4-1, one photo is a single row of “height × width × 3” cells. Click cells in the Pixel Magnifier to check the actual numbers, and change the resolution to see how the count of numbers changes.
3Pixel Distance
The Pixel Distance Trap — Shift One Cell and It’s a “Different Photo”
If you measure distance by subtracting the numbers at the same positions in two photos, can you measure “resemblance”? We shift or darken the same chihuahua and measure the distances among all five photos.
1Shift One Cell and It Becomes a “Different Photo” For the photos on this page
The pixel distance between two photos is found by subtracting the grayscale numbers at the same positions, squaring, averaging, and taking the square root (0 means identical). Try shifting the chihuahua sideways or darkening it. To the human eye it’s still the same chihuahua, but in numbers it can become a photo farther away than the muffin.
2Pixel Distance Table: Who Is the Chihuahua Close To? For the photos on this page
We measured the pixel distance between every pair of the five photos on this page. The smaller the number, the more “alike” they were computed to be. The smallest value is highlighted.
The values in this table were measured on the five photos on this page (cropped to the face and reduced to 56 × 56). With other photos, the values vary greatly with background, lighting, and framing, but the conclusion that “pixel distance differs from distance in meaning” is a well-known property of real photos too.
3Results of Colab Code 1 · Measuring with Other Photos
Code 1 runs the same experiment on the face regions of a cat photo (chelsea) and a human photo (astronaut) included in scikit-image, reduced to 56 × 56. Shifting the same cat by 1 cell raised the distance to 16.9, by 2 cells to 27.1, and by 8 cells to 40.5, while the distance between the cat and the human was 85.3. Shifting by just a few cells gets you close to half the distance to a different animal.
Pixel distance only measures “whether the numbers at the same positions are similar.” So it is sensitive to position, brightness, and framing, and it knows nothing about what is in the picture. This limitation is why we need the filters of Section 4 and the CNN of 4-3.
Explained for Beginners
The k-NN in 4-1 asked its “nearest neighbors.” To do the same with photos, the distance between photos has to reflect meaning. But by pixel distance, the chihuahua is closer to the muffin than to a fellow dog (the golden retriever), and the same chihuahua shifted by just 4 cells ends up farther away than the muffin. This means you shouldn’t find neighbors using raw pixel distance between photos.
4Convolution
Convolution and Filters — Finding Patterns Made by Neighboring Pixels
Slide a 3 × 3 table of numbers (a filter) across the photo, multiplying and adding. Places that resemble the pattern give large numbers, and collecting those results gives a feature map.
1Model
A feature is a pattern made by many pixels together, such as “bright here, then suddenly dark (an edge),” “a round spot darker than its surroundings (a spot),” or “dense thin lines (fur texture).” A CNN places a small table of numbers such as 3 × 3, called a filter (kernel), on the photo and slides it one cell at a time, multiplying the overlapping numbers and adding them all up. Places that resemble the filter’s pattern produce large numbers, other places produce small ones, and the new matrix collecting these results is a feature map. This calculation is called convolution.
InputPixel matrix X
ToolSmall filter K (e.g., 3 × 3)
OutputFeature map F
Put simplyIt’s a game of matching a pattern stamp. Press a stamp carved with a “dark on the left, bright on the right” pattern at every spot on the photo, and write a high score wherever the pattern lines up well. Once the score sheet is full, you can see at a glance where the vertical edges are in the photo. Change the stamp, and the pattern you’re looking for changes too.
2Formula
(X ∗ K)(i, j) = Σu=0…2 Σv=0…2 K(u, v) · X(i+u, j+v) + bIn mathematics, convolution flips the filter before multiplying, but in deep learning this unflipped calculation (cross-correlation) is conventionally called “convolution.” Since the filter numbers are found by training, the result is the same.
F(i, j) = ReLU((X ∗ K)(i, j)) = max(0, (X ∗ K)(i, j))ReLU turns negative values into 0. It discards “places where the pattern appears reversed” and keeps only “places where the pattern is.”
Max pooling: P(i, j) = max { F(2i, 2j), F(2i+1, 2j), F(2i, 2j+1), F(2i+1, 2j+1) }Keeps only the largest value in each 2 × 2 block, halving the width and height. The result wobbles less when a feature moves a cell or two.
K(u, v)
The 9 numbers of the filter. They play the same role as a perceptron’s weights w and are what training finds
X(i+u, j+v)
The 9 pixels overlapped under the filter
b
Bias. One per filter
F, P
Feature map, pooled feature map
Reading the Formula Piece by Piece
Σ Σ K·XMultiply the 3 × 3 = 9 pairs position by position and add them all up. It is exactly the same calculation as the perceptron’s weighted sum w · x from 4-1; the only difference is that the input is a small patch of the photo.
(i, j)The position where the filter is placed. By changing i and j, the same filter sweeps the whole photo. Because the same weights are reused at every position, an eye is found the same way wherever it is in the photo.
max(0, ·)The activation function ReLU. Its bend is what makes stacking layers meaningful.
max { 4 cells }Max pooling. A 56 × 56 feature map becomes 28 × 28, cutting the computation to a quarter.
Try It with Numbers (right rim of the chihuahua’s eye, a 3 × 3 grayscale patch)X = [92 129 174 ; 38 74 137 ; 55 119 176], vertical edge filter K = [−1 0 1 ; −1 0 1 ; −1 0 1] Multiply and add: (−92 + 174) + (−38 + 137) + (−55 + 176) = 82 + 99 + 121 = 302 → ReLU(302) = 302 (strong edge) For a flat patch of fur only, [175 176 175 ; 174 175 176 ; 175 175 174]: (175 − 175) + (176 − 174) + (174 − 175) = 1 → nearly 0 (no edge)
3Diagram · Stamping a Filter Once
4Interpretation
A filter is the “pattern you want to find” written down as numbers. A filter whose left column is −1 and right column is +1 produces larger numbers the brighter the right side is compared with the left. That’s why the vertical edge between the chihuahua’s eye (dark) and fur (bright) gives a large value like 302, while a flat area of fur gives a value close to 0.
The key point is that the same filter is reused at every position in the photo. A single filter has only 9 numbers, yet it finds edges anywhere in the photo. It also eases the problem from Section 3, where “shifting by one cell changes every number.” When the photo shifts, the feature map just shifts along with it; the fact that an edge is there stays the same.
The first layer of a real CNN uses dozens of filters like this. Each filter produces one feature map, so the output becomes a thick matrix of “photo size × number of filters.”
5Animated Diagram · A Filter Sliding to Fill a Feature Map
We place a vertical edge filter, one cell at a time, on an 8 × 8 grayscale patch near the chihuahua’s eye (cut from the photo reduced to 28 × 28). Blue numbers mean “the right is brighter,” and orange numbers mean “the left is brighter.”
Place filter
Multiply 9 pairs
Add them all
Move one cell
Clear negatives with ReLU
6Filter Lab · Try Different Stamps
Choose a photo and a filter. You can also edit the numbers in the filter cells directly. Bright areas in the feature map on the right are “where the pattern was found.” Click a cell in the feature map to see the calculation for that position below. To keep the output at 56 × 56, the area outside the edges is filled by copying the nearest edge pixel. This differs from the zero padding in Diagram 3, so values in the border cells may differ from what zero padding would give.
Photo
Filter
Input (grayscale 56 × 56)Feature map (click to compute)
Filter K (3 × 3)
7Key Features
Strengths
One filter (9 numbers) is reused across the whole photo, so there are very few weights
If the photo shifts a little, the features simply shift with it, so the decision stays steady
Stacking layers naturally builds a hierarchy: lines → parts → whole
Limitations
A filter only looks at nearby pixels, so relationships between distant parts can only be learned through many layers
On its own it handles rotation and size changes poorly, so it must be trained on varied photos
Features in early layers can’t tell similar patterns apart (eyes and blueberries)
Where it fits: in 1980, Fukushima’s Neocognitron first proposed the “local filters + stacked layers” structure, and LeCun’s LeNet (1989, 1998) trained it with backpropagation to read handwritten digits. In 2012, AlexNet won large-scale photo classification by a wide margin, opening the deep learning era.
✓Check Question
Q1. In convolution, how is the value at one filter position computed?
Q2. What do you get when you apply the vertical edge filter [−1 0 1]×3 to a flat patch where every cell is 10?
Q3. What size do you get after applying 2 × 2 max pooling to a 56 × 56 feature map?
Q4. Who decides the numbers in a CNN’s filters?
Explained for Beginners
Convolution repeats the weighted sum of the 4-1 perceptron (w·x + b) on every small patch of a photo. The 9 numbers in the filter are the weights, and because the same filter is reused at every position in the photo, an eye is found the same way no matter where it is. In Code 2, you check that the hand calculation matches scipy’s result, and you see that when handwritten digits are shifted by 2 pixels, a neural network that uses the raw pixels collapses to 14.2% while a small CNN holds on at 90.4%.
5Human-Chosen Features
Human-Chosen Features — Face Boxes, Landmarks, Edge Strength
Before deep learning, people decided “what to measure.” Here we look at the strengths of that approach and at its limit: change the photo, and the judgment flips.
1A History of Human-Chosen Features · Face Detection and Landmarks
Before deep learning, face recognition relied on human-chosen features. In 2001, Viola and Jones found face boxes in photos by rapidly computing thousands of simple rectangular features (Haar features) that measure brightness differences such as “the eye region is darker than the cheeks” [R4]. In 2005, HOG found the outlines of people by counting edge directions in each small cell [R5]. Facial landmarks, which locate the corners of the eyes, the tip of the nose and the jawline like the red dots in the Section 1 image, are then computed inside the face box found this way [R6].
These methods are fast and easy to explain, but a person has to know in advance what to measure, and they struggled with lighting, angle and occlusion. CNNs overcame this limit by learning the same kind of filters from data [R7].
2Human-Chosen Features vs Features the Model Learns
The image from Section 1. The green box is the detection result (“there is a face here”), the red dots are facial landmarks around the jawline, eyebrows and lips, and the pink dots mark the key positions of the two eyes, nose and mouth. The red lines on the cat form a facial skeleton linking the forehead, eyes and nose.
Human-chosen features
A person decides first what to measure, such as “the distance between the eyes” or “the angle of the ear tips.”
Easy to explain the reason
Works even with little data
Weak against lighting, angle and occlusion
Can’t use clues no one thought of
Features a CNN learns
The filter numbers K start out random, and training adjusts them toward the correct answer.
Finds even fur texture, bread texture and a shiny nose on its own
Fairly robust to changes in lighting and angle
Needs thousands to millions of photos
Hard to explain why it decided that way
①Cat vs Human Easy
Switch to the “Edge strength” view. Interestingly, the average edge strength over the whole photo is almost the same for both. That’s because the human photo also has many fine lines in the long hair. The difference is where the edges are. The cat has fur texture and stripes packed right into the middle of its face, while the human has smooth cheeks and forehead, so the edges gather at the eyes, nose, mouth and hair. The “center region” value below shows this difference. This is why we need a map by location, that is, a feature map, rather than a single average number.
Part features at around layer 3 also clearly separate them. Pointed triangular ears on top of the head, stripes on the forehead, a small pink nose and long whiskers don’t exist on a human face. The red lines and dots in the Section 1 image are a way for people to specify such clues by hand; a CNN finds the same clues on its own with filters.
In the table in Section 3, the grayscale pixel distance between cat and human is about the same as the distance between cat and dog. For people, this is the easiest matchup, yet pixel distance doesn’t reveal the difference. It’s one more piece of evidence that pixel distance doesn’t know meaning.
3Results of Colab Code 1 · A Single Edge-Strength Feature Flips from Photo to Photo
In the pair comparison above, using this page’s photos, the cat had the larger central edge strength. But when you measure it in Code 1 with the cat and human photos from scikit-image, it comes out the other way around (edge strength in the face area: cat 101.5, human 123.6). That’s because the hair and the teeth of the smiling mouth in the human photo create strong edges.
A single human-chosen feature like “lots of edges means cat” flips easily when the photo changes. So we need a combination of many features rather than just one, and a method that learns that combination from data. That is the topic of 4-3.
Explained for Beginners
Human-chosen features make it easy to explain “why the decision was made,” and they work even with little data. On the other hand, people have to know in advance what to look for, and they break down easily when the photo conditions change. In the pair comparison, switch to the “Edge strength” view and compare where the edges gather for the cat and the human.
6Deeper Layers
The Deeper the Layer, the Bigger What It Finds
First-layer filters find lines and edges. Stack layers, and lines combine into corners, corners into eyes and ears, and eyes and ears into a face.
1The Deeper the Layer, the Bigger What It Finds
When you stack layers while shrinking the photo with pooling, one cell in a deeper layer sees a wider area of the original photo. So early layers find small lines, deeper layers find eyes and ears, which are combinations of lines, and even deeper layers find the whole face. This figure is a simplified rendering of the tendency Zeiler and Fergus (2014) showed by visualizing the layer-by-layer responses of a trained CNN.
Explained for Beginners
When you stack layers while shrinking the photo with pooling, one cell in a deeper layer sees a wider area of the original photo. So early layers find small lines, and deeper layers find combinations of those lines [R8]. In 4-3, we’ll see how a CNN built by stacking these layers produces probabilities, and why stacking layers separates what a single plane couldn’t.
A
Colab Code — Run This Page Yourself
Both code samples use only libraries that come preinstalled in Colab and built-in data. Click “Download notebook (.ipynb)” and open it in Colab with File → Upload notebook, or copy the code and paste it into a new notebook cell.
Code 1 · Opening a Photo as a Matrix of NumbersLinked to Sections 2, 3 and 5 · numpy, scikit-image, scipy, matplotlib (photos built in) · a few seconds on CPU
Looks at the shape and pixel values of the cat and human photos included in scikit-image, and computes grayscale luminance, pixel distance and edge strength. To use your own data, read your photo with io.imread and change the (top, left, side) in small() to the position of the face.
Output (run on 2026-10-04, numpy 2.4 · scikit-image 0.26 · scipy 1.17 · CPU). There is no randomness, so it’s the same every time you run it.
Code 2 · Convolution by Hand, and Is a CNN Really Robust to Shifts?Linked to Section 4 · numpy, scipy, tensorflow (Keras), scikit-learn · about 1 minute on CPU
Computes a convolution with a 3 × 3 vertical edge filter by hand and checks it against scipy, then compares an MLP and a small CNN on test images of handwritten digits shifted by 1–2 pixels.
Output (run on 2026-10-04, TensorFlow 2.21 · Keras 3 · CPU). Average of 2 seeds. The numbers may vary slightly with the seed, version and hardware.
B
AI Prompts to Try — Use This Code as a Starting Point
These are prompts to paste into an AI assistant such as ChatGPT, Qwen, or Claude. Click “Copy” and the code you selected is attached after the prompt. Replace the text inside 〔 〕 with your own situation.
Code to attach
An AI’s answer is only a starting point. Always check any papers, figures, or library functions it suggests against the originals and the official documentation. If you used AI help for a competition entry or a paper, disclose that you did and to what extent, following the rules of the organizers or the journal.
C
Glossary
This is the same dictionary as the yellow dotted underlines in the text. The same terms are used throughout the series. Click a section name to jump to where the term first appears.
D
Analogies — and Where the Analogy Breaks Down
Analogies are a first step toward understanding, but they can also plant misunderstandings. Analogies to the human eye and brain are marked by strength. Direct There is evidence that it actually does the same thing · Partial Only some properties are similar · None No known counterpart
Concept
Everyday analogy
Human eye/brain analogy
Where the analogy breaks down
Pixel
A single mosaic tile
Partial The retina of the eye also has densely packed light-sensing cells
Retinal cells aren’t laid out evenly in a grid; they are concentrated in the center (the fovea), and the kinds and proportions of color cells differ
Convolution filter
A stamp carved with a pattern
Partial Simple cells in the visual cortex respond strongly to edges in a particular direction
Brain cells have far more complex learning and wiring. A CNN filter reuses the same numbers at every position, but the brain doesn’t
Feature map
The X on a treasure map
None
A map is a single sheet, but a real layer has as many feature maps stacked up as it has filters
Pooling
Picking a class representative
None
Class representatives are chosen by people, but max pooling mechanically picks the cell with the largest number
Facial landmarks
Signposts on a map
None
Signposts are put up by people, but landmark positions are computed by a trained model for each photo. If part of the face is covered, they land in the wrong spot
E
About the Author and References
E-1 About the Author · E-2 Academic References · E-3 Figure and Data Sources
E-1 About the Author
Sangsun Ahn (CEO, M-Robo Inc.)
After working at the Korea Electrical Safety Corporation and the Maeil Business Newspaper, he founded M-Robo Inc., an external venture of Maeil Business Newspaper, where he currently serves as CEO. M-Robo Inc. is a media venture that began as an in-house venture of a news company. It provided a robot reporter service (StockBot) to the Kukmin Ilbo and has carried out numerous projects for organizations including the Korea Internet & Security Agency, the Korea Press Foundation, and the Korea Radio Promotion Association. It now runs its own AI news service, Money-Robo. While serving as CEO, he won the Asia Open Data Hackathon and received the Minister of the Interior and Safety Award twice for contributions to data entrepreneurship, and he is the inventor and registrant of five patents in artificial intelligence and blockchain. He is the author of the book “AI Crime and Self-Defense”, and he has developed e-learning content on fintech, statistics, and artificial intelligence for the Online Lifelong Education Institute of Korea University of Technology and Education. He holds a Ph.D. in economics (Dongguk University, mathematical microeconomics). He teaches data and AI courses at several universities, including Kookmin University and Hoseo University. Full profile
Courses
Ahn, Sangsun (2025). “Data Science for Business” (course materials). Department of AI Convergence, Open Cyber University of Korea.
Ahn, Sangsun (2025). [K-Digital] AI Natural Language Processing. STEP (Smart Training Education Platform), Online Lifelong Education Institute, Korea University of Technology and Education.
Ahn, Sangsun (2024). [K-Digital] Fintech Data Analysis Basics. STEP (Smart Training Education Platform), Online Lifelong Education Institute, Korea University of Technology and Education.
Ahn, Sangsun (2025). A study on household expenditure prediction models using the Korean Retirement and Income Study panel data: An AI model-based approach. “Future Society”, 16(2). Full text
Ahn, Sangsun (2025). A study on household expenditure prediction models using the Korean Retirement and Income Study panel data: An AI approach. Presentation at the KReIS Panel Conference (2025.05.13), National Pension Research Institute, National Pension Service. Slides
Ahn, Sangsun (2025). A study on an algorithm for detecting suspected drug transactions from SNS text: Focusing on pattern and context analysis. “Future Society”, 16(1). Full text
Ahn, Sangsun (2024). Statistics-based countermeasures against adversarial data for securing anomaly detection models in finance: Empirical analysis and simulation results. “Future Society”, 15(3). Full text
Ahn, Sangsun (2024). An AI-based solution for detecting SNS posts based on virtual character traits. Poster presentation at the 2024 Summer Newmedia Party (2024.06.19), Seoul Media Institute of Technology.
Ahn, Sangsun (2024). Machine learning models using industrial accident panel data. Poster presentation at the 10th Panel Study of Workers’ Compensation Insurance Conference (2024.05.17), Labor Welfare Research Institute, Korea Workers’ Compensation & Welfare Service.
Ahn, Sangsun (2024). Analysis and forecasting of virtual asset time-series data: Comparing time-series and AI models and suggesting improvements. “Future Society”, 15(1). Full text
E-2 Academic References
[R1] ITU-R Recommendation BT.601. Studio encoding parameters of digital television. — Section 2 (grayscale luminance coefficients)
[R2] LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324. — Section 4
[R3] Fukushima, K. (1980). Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36, 193–202. — Section 4
[R4] Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. CVPR 2001. — Section 5
[R5] Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. CVPR 2005. — Section 5
[R6] Kazemi, V., & Sullivan, J. (2014). One millisecond face alignment with an ensemble of regression trees. CVPR 2014. — Sections 1 and 5
[R7] Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. NeurIPS 2012. — Section 5
[R8] Zeiler, M. D., & Fergus, R. (2014). Visualizing and understanding convolutional networks. ECCV 2014. — Section 6
[R9] van der Walt, S., et al. (2014). scikit-image: image processing in Python. PeerJ, 2, e453. — Code 1 (chelsea and astronaut photos)
E-3 Figure and Data Sources
The image in Section 1 was made by the author. The cat and human photos used in the hands-on activities were cropped from the face areas of this image, reduced to 56 × 56, and had the lines and dots removed
The chihuahua photo in Section 2 was cropped from the 4-4 image
The layer-by-layer figure in Section 6 is a conceptual diagram that simplifies the tendency shown in [R8]
F
TIP for Educators
A guide for instructors, on-the-job training (OJT) leads, and tutors. Times are estimates, so adjust them to your actual class.
Audience
College students and working professionals who have finished 4-1. Practitioners who want to know “how image AI receives a photo”
Prerequisites
Features, targets and weighted sums from 4-1; multiplication and addition
Learning Goals
① Calculate the shape of a color photo (H × W × 3) and how many numbers it contains ② Compute one 3 × 3 convolution cell by hand ③ Explain the limits of pixel distance and human-chosen features with examples
Estimated Time
Reading 40–50 min · hands-on demo 25 min · Colab practice 25 min (Code 1: 10 min; Code 2: 1 min to run + 14 min to interpret)
Three Lesson Plans
50-min lecture
Concept-focused
Intro 5 min (Section 1) → Section 2 Pixel Magnifier 15 min → Section 3 shift experiment 10 min → Section 4 convolution by hand 15 min → Check Questions 5 min.
90-min on-the-job training
Work-focused
Use Section 2 to calculate the resolution and number count of photos from your own work (defective products, scanned documents, etc.) 20 min → find filters that suit your photos in the Section 4 Filter Lab 25 min → discuss the limits of human-chosen features from Section 5 15 min → run and interpret Code 2 30 min.
1:1 tutoring
Paced by level
Beginners: Sections 1, 2 and 4 only. Have intermediate learners redo the output-size formula in Section 4 with padding and stride, and predict how robustness to shifts changes if GlobalMaxPooling is replaced with Flatten in the Code 2 CNN.
Teaching Points by Section
Section
Questions to pause and ask
Interaction tips
1 Problem
“What do people look at to know it’s a cat?”
Point out that the lines and dots are clues chosen by people
2 Pixels
“What are the three numbers in the eye cell?”
Change the resolution in the Pixel Magnifier from 7 → 56
3 Pixel distance
“It’s the same chihuahua, so why is it farther than the muffin?”
Move the shift slider to 4 pixels or more
4 Convolution
“What happens if you stamp a vertical edge filter on a flat patch of fur?”
Pause the animation at “Add them all” and check it by hand
5 Human-chosen features
Is “lots of edges means cat” always true?
Look at the edge-strength view in the pair comparison together with the Code 1 results
6 Deeper layers
“Which layer holds the filter that finds eyes?”
Walk through the layer-by-layer figure from left to right
Top 5 Common Misconceptions
Misconception
Correct explanation
Scene to show
A photo is a picture file, so it’s different from numbers
To a computer, a photo is H × W × 3 numbers from 0 to 255
Section 2 Pixel Magnifier
Similar pixels mean the same thing was photographed
Shifting by just one pixel changes the distance a lot. Pixel distance doesn’t know meaning
Section 3 shift experiment
Convolution is complicated math
It just multiplies 9 overlapping pairs and adds them up. Same as the perceptron’s weighted sum
Number example in Section 4
Filters are designed by people
The filters in the hands-on activity are examples. A CNN learns the filter numbers from data
Section 4 Check Question Q4
Human-chosen features are useless
They’re easy to explain and need little data. But they’re weak when conditions change
Section 5 and the flipped result in Code 1
Check Questions (click to expand the explanations)
Q1. How many numbers are in a 56 × 56 color photo? In grayscale?
Q3. What do you get when you stamp the vertical edge filter [−1 0 1]×3 on the patch [92 129 174; 38 74 137; 55 119 176]?
(174−92) + (137−38) + (176−55) = 82 + 99 + 121 = 302. A strong edge, brighter on the right.
Q4. What is the output size when a 3 × 3 filter is applied to a 56 × 56 input without padding (stride 1)?
56 − 3 + 1 = 54, i.e., 54 × 54. With padding 1, it stays 56 × 56.
Q5. In Code 2, the MLP scored 14.2% on digits shifted by 2 pixels, and the CNN scored 90.4%. What two things made the difference?
Convolution, which reuses the same filter at every position (finding the same pattern even when its position changes), and global max pooling, which throws away position and keeps only the largest response.
Colab Assignments
Basic
In Code 1, read a photo of your own face (or your pet), print its shape and number count, and make a table of pixel distances for shifts of 1, 2 and 4 pixels.
Challenge
In the Code 2 CNN, replace GlobalMaxPooling2D with Flatten, retrain, and report the change in accuracy on photos shifted by 1 and 2 pixels using 3 seeds. Explain why, linking it to the interpretation in Section 4.
Adjusting Difficulty and Learning More
Beginners
You can skip the distance table in Section 3 and Section 6. Just make sure to do the hands-on activities in Sections 2 and 4
Advanced
Have them research Haar features and HOG, then build similar filters in the Section 4 Filter Lab. Check the papers they research in the original
Learn more
3-3 Images and CNNs · 4-3 Cat vs Dog (overall CNN structure)
