An Introduction to Computer Vision & Image Processing

What is Computer Vision?

The goal: to make computers understand and analyze visual content the same way humans do.

Three complementary one-line definitions to memorize:

• Vision is automating human visual processes. الرؤية هي أتمتة العمليات البصرية البشرية.
• Vision is an information-processing task. الرؤية مهمة معالجة معلومات.
• Vision is inverting image formation. الرؤية هي عكس عملية تكوين الصورة — أي استرجاع معلومات المشهد من الصورة ثنائية الأبعاد، وقد يشمل ذلك البنية ثلاثية الأبعاد.

The four sub-areas of Computer Vision

The field decomposes into four canonical problems — memorize them in order, since later courses follow the same progression:

1. Imaging — how images are formed (pinhole, lenses, sensors). التصوير: كيف تتشكّل الصورة.
2. Features and boundaries — extracting edges, corners, regions. السمات والحدود: استخراج الحواف والزوايا والمناطق.
3. 3D reconstruction — recovering 3D structure from 2D images. إعادة البناء ثلاثي الأبعاد: استرجاع البنية ثلاثية الأبعاد من صور ثنائية الأبعاد.
4. Visual perception — high-level recognition, scene understanding. الإدراك البصري: التعرّف والفهم على مستوى عالٍ للمشهد.

Why build machines that emulate human vision?

Given that the human visual system is so powerful (about 60% of the brain is involved in visual processing!), why bother building artificial ones?

1. To free up human time for more rewarding activities.
   لتحرير الوقت لأنشطة أكثر فائدة.
2. Our vision is not good at precise measurement of the physical world.
   رؤيتنا ليست جيدة في القياسات الدقيقة.
3. Machines can surpass human vision — extracting information we simply cannot see (infrared, satellite imagery, microscopy, etc.).
   الآلات يمكنها تجاوز قدرة الإنسان واستخراج معلومات لا نراها.

First principles vs. just throwing data at a neural net

A natural question: why not just train a neural network with tons of data? The lecture gives three pointed answers:

1. It's unnecessary to train a network for a phenomenon that can be precisely described from first principles. لا داعي لتدريب شبكة على ظاهرة يمكن وصفها رياضياً بدقة من المبادئ الأولى.
2. When a network fails, first principles are the only hope for understanding why. عند فشل الشبكة، المبادئ الأولى هي الأمل الوحيد لفهم السبب.
3. Models based on first principles can be used to synthesize training data instead of laboriously collecting it. يمكن للنماذج المبنية على المبادئ الأولى توليد بيانات التدريب بدل جمعها.

A typical computer-vision pipeline

Memorize this four-stage structure — it appears in almost every exam:

Vision is multi-disciplinary

Computer vision lives inside Artificial Intelligence and overlaps with: Machine Learning, Optics, Robotics, Image Preprocessing, Domain Knowledge, and NLP.

Real-world applications

You don't need to memorize all of these, but recognize the categories — exam questions often ask "give 3 applications of CV".

How do humans do it? — Inside the visual brain

Vision is easy for us — but we don't fully understand how we do it. About 60% of the brain is involved in visual processing, distributed across many specialized regions:

Note: A V6 region has been identified only in monkeys, not yet in humans.

Vision Research — illusions, ambiguities, and the limits of "seeing"

Three statements summarize the state of vision research — they often appear as a short-answer exam item:

1. Vision is a hard problem. الرؤية مسألة صعبة.
2. Vision is multi-disciplinary (AI, ML, optics, robotics, NLP, image processing, domain knowledge). الرؤية متعددة التخصصات.
3. Considerable progress has been made with many successful real-world applications. تحقّق تقدّم كبير مع تطبيقات ناجحة في العالم الحقيقي.

Human vision is fallible

Human vision is more fallible than we may like to believe. We see what we see and believe it to be accurate.
الرؤية البشرية أكثر عرضة للخطأ مما نحب أن نعتقد — نرى ما نرى ونؤمن بأنه دقيق.

Vision research relies heavily on illusions to expose how the brain (and any vision system) infers rather than simply records. Key examples to know:

• Fraser's Spiral — appears to be a spiral; is actually concentric circles. حلزون فريزر يبدو حلزونياً وهو في الحقيقة دوائر متراكزة.
• Checker Shadow (Adelson) — square A and square B have the same brightness in pixels, but B looks brighter because the brain "discounts" the cylinder's shadow. المربعان A و B بنفس السطوع، لكن الدماغ يصحّح للظل.
• Donguri Wave — perceived motion without motion.
• Forced Perspective — two people of equal height made to look very different.

Visual ambiguities · الغموض البصري

Different from illusions: ambiguities are images that genuinely admit multiple valid interpretations.

• Six cubes or seven? — depending on which faces you treat as "front".
• Young-girl / old-woman, face / vase — figure-ground reversal.
• Crater on a mound / mound in a crater — flipping the image reverses the perceived depth, because our brain assumes light comes from above.

Seeing vs. Thinking — the Kanizsa Triangle

You see a white triangle that isn't physically there. Your visual system fabricates contours from the Pac-Man-like inducers. This is thinking bleeding into seeing.

Image Formation

Definition. An image is the projection of a 3D scene onto a 2D plane. To understand vision, we must understand the geometric and photometric relationship between the scene and its image.

Three sub-topics structure this lecture:

1. Pinhole and perspective projection.
2. Image formation using lenses.
3. Lens-related issues.

The Pinhole Camera

Three key definitions you must be able to state precisely:

• Pinhole · الثقب الضيق: an opaque sheet with a tiny hole in it.
• Optical axis · المحور البصري: an axis perpendicular to the image plane (passing through the pinhole).
• Effective focal length, f · البعد البؤري الفعّال: the distance between the pinhole and the image plane.

Camera Obscura — the historical pinhole · الغرفة المظلمة

The pinhole camera is the same idea as the camera obscura (literally "dark chamber"). A pinhole in one wall of a darkened room projects an image of the outside scene onto the opposite wall — geometrically accurate enough that Renaissance artists used it as a tracing aid.

Perspective Projection — the equations

Set up coordinates with the origin at the pinhole. Let:

• Scene point P₀ at position r̄₀ = (x₀, y₀, z₀)
• Image point Pᵢ at position r̄ᵢ = (xᵢ, yᵢ, f) on the image plane.

By similar triangles (the ray from P₀ to Pᵢ passes through the origin):

Component-wise, this gives the celebrated perspective projection equations:

Or equivalently, solving for the image coordinates:

Perspective projection of a line

Question: what is the perspective projection of a 3D line onto the image plane?

Answer: The image of a 3D line is a line in 2D. Straight lines in the scene map to straight lines in the photograph.

Image Magnification

Consider a tiny segment A₀B₀ in the scene that lies on a plane parallel to the image plane (so all of it shares the same depth z₀). Let:

• A₀ at (x₀, y₀, z₀)
• B₀ at (x₀+δx₀, y₀+δy₀, z₀) — displaced by (δx₀, δy₀).
• Their images: Aᵢ=(xᵢ, yᵢ), Bᵢ=(xᵢ+δxᵢ, yᵢ+δyᵢ).

From perspective projection applied to both endpoints, and subtracting:

Now magnification is the ratio of image-vector length to scene-vector length:

Substitute step ① in:

(m is negative when the image is inverted, which it physically is in a pinhole camera — but the formula is usually stated with absolute value.)

Consequence: image size is inversely proportional to depth

Closer objects look bigger; far ones look small. This is why train tracks "narrow" toward the horizon and why a forced-perspective trick (e.g. holding someone in your palm) works.

Area magnification

If linear magnification is m, then area scales as m²:

Vanishing Points

Definition. A vanishing point is the image point where a set of 3D parallel lines appears to disappear / converge.

• Parallel scene lines converge at a single image point — the vanishing point.
• The exact location of the vanishing point depends on the orientation of those parallel lines in 3D.
• Lines parallel to the image plane have a vanishing point at infinity (they stay parallel in the image).

How to compute the vanishing point — three-step recipe

1. Define the direction of the parallel lines as a vector ⟨lₓ, l_y, l_z⟩. Since the origin is at the pinhole, this vector also describes a point P on the line through the pinhole that is parallel to the family of lines.
2. Project P onto the image using the perspective projection equations.
3. The resulting image point is the vanishing point:

Use of vanishing points in art · في الفن

Vermeer's The Music Lesson (c. 1662–1664) is a textbook example: all the receding edges of the room converge on a single point — used both as a compositional anchor and as evidence that Vermeer may have used a camera obscura.

False perspective · المنظور الكاذب

A corridor in Palazzo Spada (Borromini) appears ~155 ft deep but is actually only ~30 ft. By manipulating where vanishing points fall and how columns scale, the architect tricked the perspective system. Same principle as Hollywood "forced perspective" shots.

What is the ideal pinhole size?

The lecture shows photographs taken with pinholes ranging from 2 mm down to 0.07 mm. Sharpness peaks around 0.35 mm. Why?

where f is effective focal length and λ is the wavelength of light.

Exposure time — and why we need lenses

Now the bad news: with a tiny pinhole (the optimal one), only a tiny amount of light reaches the image plane per unit time. So to get a usable, bright image, you must leave the shutter open for a long time.

The lecture's example image of the Flatiron Building is captured with f = 73 mm, d = 0.2 mm, and a 12-second exposure. Twelve seconds is fine for a still building. It's terrible for moving cars, people, or any handheld photograph.

That trade-off is the cliffhanger for the next lecture (image formation using lenses, lens-related issues).

Lenses: brightness without losing geometry

A lens performs perspective projection like a pinhole, but gathers significantly more light. The center of the lens plays the role of the pinhole, while the lens bends rays so many rays from one scene point meet again at one image point.

f is focal length, o is object distance, and i is image distance. The slide example asks: if f = 50 mm and o = 300 mm, find i. Compute: 1/i = 1/50 - 1/300 = 5/300 = 1/60, so i = 60 mm.

In a two-lens system, the first lens forms an intermediate image and the second lens images that intermediate result. The total magnification is the product of the individual magnifications.

Defocus, blur circle, and depth of field

A lens is focused for one object distance. A point away from the plane of focus does not land as a point on the sensor; it becomes a blur circle. The farther the point is from the focus plane, the larger the blur circle.

Depth of field is the range of object distances over which the image is "sufficiently well focused"; in the lecture, that means the blur is smaller than the finite pixel size C.

Depth-of-field formulas and hyperfocal distance

Let c be acceptable blur size, usually tied to pixel size. The near and far acceptable object distances are o₁ and o₂:

Tilted lenses and the plane of focus

Normally the lens plane, sensor plane, and focus plane are parallel. With a tilt-lens camera, the plane of focus can be tilted to match a slanted object, such as a model train. This is the idea behind Scheimpflug adapters and tilt-shift photography.

Lens-related issues

Real lenses are imperfect. It is challenging to create an image with the same quality across the whole image plane, so lenses often need compound designs: multiple lens elements with different shapes and materials compensate for one another's defects.

From optical images to digital numbers

Image sensing converts an optical image into a digital image so a computer can store, process, and analyze it.

• Resolution: number of pixels in the image.
• Noise: undesirable modifications to the image signal.
• Dynamic range: range of brightness values that the sensor can measure.

Film: photochemical sensing

Black-and-white film has four layers: protective coat, gelatin with silver halide crystals, polymer film base, and anti-halation backing. Exposure creates a latent image: invisible after exposure, visible only after development.

Silicon sensors: photoelectric conversion

A photon with sufficient energy striking silicon creates an electron-hole pair. The sensor collects the generated charge, then electronics convert that charge into voltage and finally into digital numbers.

Once pixel size becomes comparable to the wavelength of light, making pixels smaller no longer improves true optical resolution; diffraction and optics become the limit.

CCD vs. CMOS sensors

How does a sensor measure color?

A bare pixel cannot know color; it only converts photons to electrons. To measure color, a color filter array places red, green, or blue filters over pixels. Since each pixel measures only one color, the missing channels are estimated by interpolation after capture.

Noise in image sensors

Noise is any unwanted modification of the signal during capture, conversion, transmission, processing, or storage. The lecture groups sensor noise into scene-dependent and scene-independent sources.

Binary images, thresholds, and moments

A binary image has only two values: 0 or 1. It is easy to store, process, and analyze. A common way to create it is thresholding:

Image moments

Image moments are statistical parameters used to describe segmented objects. They give simple geometric properties such as area, centroid/position, and orientation.

For orientation, the lecture represents a line as:

where θ is the line angle and ρ is the perpendicular distance from the line to the origin. The second moment E is the integral/sum of squared perpendicular distances from object points to the chosen axis; choose the axis that minimizes E.

with shifted coordinates x' = x - x̄, y' = y - ȳ, where a = ∬x'²b, b = 2∬x'y'b, and c = ∬y'²b. The orientation axis passes through the centroid.

Roundedness is close to 1 for compact/circular objects because spread is similar in every direction. It becomes smaller for elongated objects because the minimum and maximum moments differ strongly.

Connected components, topology, and skeletons

Connected-component labeling

When a binary image has multiple objects, segment it into separate components by assigning labels to connected foreground pixels.

Neighborhood and connectedness

The meaning of "neighbor" matters. 4-connectedness uses the horizontal and vertical neighbors; 8-connectedness also includes diagonals. The lecture also mentions hexagonal tessellation as a 6-connected arrangement.

Topology and Euler number

Topology studies shape properties preserved by stretching or bending, but not tearing or gluing. For binary shapes, the Euler number is commonly used as:

Skeletons

A skeleton is a simplified centerline representation of a binary shape. It preserves the essential structure while removing boundary thickness.

• Shape simplification.
• Feature extraction from vessels, nerves, roots, and similar structures.
• Shape matching and handwriting recognition.
• Path planning and navigation.
• Topological analysis of biological networks.

Images as functions and convolution systems

An image can be treated as a function f(x,y). Point processing applies a transformation T independently at each pixel to produce g(x,y). Examples include brightness/contrast changes and thresholding.

Why images become grainy or blurry

• High ISO: amplifies the signal in low light and increases visible grain.
• Low light: too few photons arrive, so noise becomes noticeable.
• Small sensor/pixels: smartphone pixels collect fewer photons and are more noise-sensitive.
• Blur: can come from defocus, motion, or smoothing operations.

Linear shift-invariant systems (LSIS)

An LSIS is both linear and shift invariant. In imaging, ideal lenses and many filters are modeled this way.

Commutativity lets you swap image/filter order mathematically. Associativity means cascaded filters can be combined first: filtering by b and then c is equivalent to one filter b*c.

The unit impulse or delta function leaves a function unchanged when convolved with it. The response of a system to an impulse is its impulse response. In imaging, this is called the point spread function (PSF): how a point source spreads on the image/retina.

Discrete 2D convolution

Convolution with discrete images slides a kernel over the image and computes weighted sums. Borders are a practical issue: common solutions are ignoring borders, padding with a constant value, or padding by reflection.

Smoothing filters: box, Gaussian, median, bilateral

Gaussian smoothing is separable: a 2D Gaussian filter can be applied as a 1D horizontal filter followed by a 1D vertical filter. For a K×K kernel, direct 2D convolution costs about K² multiplications per pixel; separable filtering costs about 2K.

The first Gaussian measures spatial closeness; the second measures intensity similarity. That second term is why bilateral filtering is nonlinear and edge-preserving. Small intensity sigma rejects cross-edge pixels strongly; very large intensity sigma makes the intensity term nearly constant, so the filter behaves much more like ordinary Gaussian smoothing.

Fourier transform and frequency-domain filtering

The Fourier idea: a signal or image can be understood by asking which sinusoidal frequencies are inside it. The spatial domain asks "what value is at this pixel/location?"; the frequency domain asks "which frequencies are present, with what amplitude and phase?"

A is amplitude, u is frequency, and φ is phase. Higher frequency means faster oscillation: in images, this usually means fine detail, edges, texture, or noise.

From Fourier series to Fourier transform

The lecture builds intuition with Fourier series: a square wave can be approximated by adding sinusoids. More terms add sharper edges. The Fourier Transform generalizes this idea from periodic sums to a continuous frequency representation.

The sinusoids are inside the complex exponential. The Fourier coefficient captures both amplitude and phase, so the Fourier transform is generally complex.

Displayed spectra usually show log(|F|), because the DC/low-frequency values can dominate so strongly that weaker frequency components would otherwise be invisible.

Transform pairs to recognize

Fourier Transform properties

Convolution theorem

The important connection with Lecture 6 is two-way:

Practically, large convolutions can be computed through FT: transform the image and filter, multiply their spectra, then apply IFT. Conceptually, this also explains filtering: a blur kernel is a frequency mask that suppresses high frequencies.

Gaussian smoothing in frequency space

Spatial view: convolve the signal/image with a Gaussian kernel. Frequency view: multiply the spectrum by a Gaussian low-pass. High-frequency noise/details shrink; low frequencies remain, so the output becomes smoother.

2D Fourier transform for images

Images have horizontal and vertical frequencies. Low frequencies describe smooth, slowly varying intensity; high frequencies describe rapid changes such as edges, texture, and noise.

يا كتكووت :v visual stories

How to use this page

This section turns the most exam-heavy ideas into small stories. Read the picture first, then say the formula out loud, then answer the “يا كتكووت” prompt in one sentence.

🐥يا كتكووت: كل صورة هنا مثل لقطة صغيرة في رأسك. تشوفيها، تبتسمي، وبعدين تقولي: آه، هذا القانون أعرف قصته.

الفكرة هنا: اربطي كل قانون بصورة قصيرة. عندما يظهر القانون في الامتحان، ارجعي للقصة البصرية ثم اكتبي المعنى.

The rule: if you can explain it from the picture in ten seconds, you probably understand it well enough to solve a question.

يا كتكووت

Pinhole vs. lens: the brightness-sharpness story

🐣يا كتكووت: تخيّلي إنك ماسكة صندوق كاميرا صغير. أول ثقب يقول لك: “أنا حاد بس نعسان، ضوئي قليل.” الثقب الكبير يقول: “أنا منوّر بس أخربط الصورة.” فتجي العدسة وتقول: “ولا يهمك، أجيب ضوء كثير وأرتبه في نقطة واحدة.”

Memory line: tiny hole = sharp but dim; big hole = bright but blurry; lens = bright and sharp when focused.

When a pinhole question appears, do not start with equations. Start with rays: one narrow ray is easy to keep sharp but carries little light; many unfocused rays are bright but smear; a lens is the useful trick because it accepts many rays and focuses them back together.

1. Pinhole: few rays → sharp but dim → needs long exposure.
2. Big hole without lens: many rays → bright but blurry because each scene point spreads.
3. Convex lens: many rays + refraction → focused point → bright and sharp when the sensor is at the right distance.

يا كتكووت

Binary shapes: from “gray mess” to measurements

🐥يا كتكووت: الصورة الرمادية داخلة الامتحان وهي متلخبطة. تحطي العتبة T كأنها بوابة: اللي يعبر يصير 1، واللي ما يعبر يصير 0. بعدها تبدئي تقيسي: وين القلب؟ بأي اتجاه مائلة؟ هل هي مدوّرة ولا ممدودة؟

Memory line: threshold first, then moments; moments are the measuring tape of a binary object.

The binary chapter is a recipe. First choose a threshold, then the object becomes a mask, then moments turn the mask into numbers: area, centroid, orientation, roundedness, and connected components.

1. Threshold: pixels above or below T become foreground/background.
2. Moments: summarize where the pixels are and how they spread.
3. Shape: roundness = small spread direction divided by large spread direction.

يا كتكووت

Fourier: image ingredients instead of pixels

🐣يا كتكووت: فورييه يقول لكِ: “لا تنظري للصورة كبكسلات فقط، اسمعي موسيقاها.” التغيّر الهادئ صوت منخفض في الوسط، الحواف نقرات أسرع، والضجيج رشّات صغيرة في الأطراف.

Memory line: center of spectrum = smooth/slow changes; outside = edges, texture, noise.

Spatial domain asks “what is the pixel here?” Fourier asks “which waves built this picture?” Smooth lighting sits near the center of the spectrum. Edges, details, texture, and noise move outward toward higher frequencies.

1. Low frequency: slow changes, background brightness, blur-friendly content.
2. High frequency: rapid changes, edges, fine texture, noise.
3. Filtering: low-pass removes high-frequency detail; high-pass removes slow background.

Formula stories يا كتكووت should remember cold

Slide visual atlas

Use this section as a fast visual review. The images are rendered from the lecture slides and paired with the exact idea to remember.

Lectures 1-2 visuals: perception and pinhole geometry

Checker shadow

Equal pixel brightness can look different because vision interprets illumination context.

Kanizsa triangle

The brain completes missing contours; seeing is interpretation, not raw measurement.

Projection equations

The core pinhole result: image position scales by focal length and divides by depth.

Vanishing point

Parallel 3D lines meet at the projection of their shared direction vector.

Lecture 3 visuals: lenses and focus

Thin lens law

Relates focal length, object distance, and image distance: 1/f = 1/o + 1/i.

f-number

f/32 is small aperture; f/5.6 is large aperture. Bigger number means smaller opening.

Blur circle

Out-of-focus scene points become disks, not points, on the sensor.

Depth of field

Acceptable focus means blur stays smaller than pixel size C.

Vignetting

Less light reaches the periphery, so corners become darker.

Chromatic aberration

Different wavelengths focus differently, creating colored edges.

Lecture 4 visuals: sensors and noise

Photon to charge

A photon in silicon creates an electron-hole pair: the basis of digital sensing.

Pixel size

Compute physical pixel pitch from sensor dimensions divided by pixel count.

CCD bucket brigade

Charges are shifted row by row, then converted to voltage and digital output.

Measuring color

Each pixel sees one filtered color; missing RGB values are interpolated.

Photon shot noise

Random photon arrivals follow Poisson behavior and depend on brightness.

Dark frame

Subtract a closed-shutter frame to remove stable dark current and fixed-pattern noise.

Lecture 5 visuals: binary shape analysis

Threshold T

The core step for converting grayscale images into binary masks.

Orientation axis

Choose the line/axis that minimizes the second moment of object pixels.

Roundedness

Compare minimum and maximum moments: E_min / E_max.

Region growing

Start from a seed and recursively label connected foreground neighbors.

Connectedness ambiguity

4-, 8-, and 6-connected choices change topology and diagonal behavior.

Sequential labeling

Raster scan, assign labels, and record equivalent labels when components merge.

Euler number

Components minus holes: a compact topological descriptor.

Skeletons

Thin a shape to its essential centerline for matching, path planning, and topology.

Lectures 6-7 visuals: filters and frequency

Convolution ⇔ LSIS

Linear shift-invariant systems are exactly convolution systems.

Convolution properties

Commutativity and associativity explain filter ordering and cascades.

Border problem

At image edges, solve missing neighbors by ignoring, constant padding, or reflection.

Gaussian kernel

A fuzzy weighted filter; nearby pixels get higher weight than far pixels.

Median filter

Nonlinear smoothing that resists outliers but can blur details if K is large.

Bilateral filter

Weights by both distance and intensity similarity, preserving edges.

Frequency view

Represent signals/images by how much of each sinusoidal frequency they contain.

Fourier series

Adding sinusoids reconstructs sharp periodic signals like a square wave.

Transform pairs

Recognize constants, deltas, sinusoids, rectangles, and Gaussians by their spectra.

FT properties

Linearity, scaling, shifting, and differentiation are common exam prompts.

Convolution theorem

Convolution in space becomes multiplication in the Fourier domain.

FT filtering workflow

Transform, multiply by filter spectrum, inverse-transform to get the filtered signal.

2D DFT

Images use horizontal and vertical frequency indices with an IDFT normalization.

Spectrum examples

Oriented structures create oriented frequency patterns; noise spreads broadly.

Low-pass

Suppresses high frequencies, so the image becomes smoother.

High-pass

Suppresses low frequencies and emphasizes edges/details.

Practice Questions

Click each question to reveal a model answer. Try to answer in your head first — the friction is the point.