Project 4: Image Warping
and Mosaicing

E Harrison  —  CS180, Fall 2026

CS180 Projects  /  Home  /  Part A Assignment

Overview

In this project I stitch photos together into panoramas. I pick matching points between overlapping photos, solve for the homography between them, warp the images into a shared frame, and blend them into a mosaic.

Part A: Image Warping and Mosaicing

A.1  Shoot the Pictures

I shot three sets of photos around campus, rotating the camera about a fixed center of projection between shots. The Gateway and Sather Gate sets pan left to right, and the Tree set pans from the ground up to the sky.

A.2  Recover Homographies

A homography maps a point (x, y) in one image to (x', y') in another. Fixing h33 = 1 leaves 8 unknowns, and each correspondence gives two linear equations:

I picked 5–6 correspondences per pair, stacked their equations into Ah = b, and solved the overdetermined system with least squares. Each mosaic uses image 2 as the reference, so I recover two homographies per set: image 1 → image 2 and image 3 → image 2.

def compute_H(im1_pts, im2_pts):
    A, b = [], []
    for (x, y), (xp, yp) in zip(im1_pts, im2_pts):
        A.append([x, y, 1, 0, 0, 0, -x * xp, -y * xp])
        A.append([0, 0, 0, x, y, 1, -x * yp, -y * yp])
        b.append(xp)
        b.append(yp)
    A, b = np.array(A), np.array(b)
    H = np.linalg.lstsq(A, b)
    return H[0]

def get_homography_matrix(im1_pts, im2_pts):
    H_vector = compute_H(im1_pts.astype(np.float64), im2_pts.astype(np.float64))
    return np.append(H_vector, 1.0).reshape(3, 3)

Below are the correspondences for each pair (matching numbers are the same point), the full system of equations (click to expand), and the recovered homography.

A.3  Warp the Images

Both warps use inverse warping. First I push the source image's four corners through H to get the bounding box of the output. Then, for every output pixel, I map it back into the source image with H−1 (dividing by the homogeneous coordinate) and sample a color there. Pixels that land outside the source are left black. Going backward means every output pixel gets exactly one color, so there are no holes.

The two methods differ only in how they sample that color. Nearest neighbor rounds the source location to the closest pixel. Bilinear takes the four surrounding pixels and blends them by distance: first along x (top and bottom pairs), then along y.

def warp_image(im, H, color_fn):
    min_x, min_y, max_x, max_y = find_warp_image_bounds(im, H)
    height = max_y - min_y + 1
    width = max_x - min_x + 1
    warped_im = np.zeros((height, width, 3))
    H_inv = np.linalg.inv(H)
    for yp in range(height):
        for xp in range(width):
            point = np.array([xp + min_x, yp + min_y, 1.0])
            point_approx = H_inv @ point
            point_approx /= point_approx[2]
            color = color_fn(im, point_approx)
            warped_im[yp, xp] = color
    return warped_im

def warp_image_nearest_neighbor(im, H):
    def nearest_neighbor(src_im, point_approx):
        height, width = src_im.shape[:2]
        x, y = np.rint(point_approx[:2]).astype(int)
        if x >= 0 and x < width and y >= 0 and y < height:
            return src_im[y, x]
        return [0, 0, 0]
    return warp_image(im, H, nearest_neighbor)

def warp_image_bilinear(im, H):
    def bilinear(src_im, point_approx):
        height, width = src_im.shape[:2]
        x, y = point_approx[:2]
        if x < 0 or x > width - 1 or y < 0 or y > height - 1:
            return [0, 0, 0]
        low_x, low_y = np.floor(point_approx[:2]).astype(int)
        high_x, high_y = low_x + 1, low_y + 1
        dx = x - low_x
        dy = y - low_y
        # colors at each corner
        top_left = src_im[low_y, low_x]
        top_right = src_im[low_y, high_x]
        bottom_left = src_im[high_y, low_x]
        bottom_right = src_im[high_y, high_x]
        # x interpolation
        top = (1 - dx) * top_left + dx * top_right
        bottom = (1 - dx) * bottom_left + dx * bottom_right
        # y interpolation
        return (1 - dy) * top + dy * bottom
    return warp_image(im, H, bilinear)

Rectification

To rectify, I clicked the 4 corners of something I know is a rectangle and computed the homography that sends them to the corners of an axis-aligned rectangle.

Nearest Neighbor vs. Bilinear

The door rectification stretches the right side of the image a lot, so it's a good place to compare. Nearest neighbor is faster but gets blocky and jagged where pixels are upsampled. Bilinear is slower (four lookups per pixel instead of one) but much smoother. For the mosaics I used nearest neighbor, since the photos are big and it ran a lot faster on my computer.

A.4  Blend the Images into a Mosaic

For each set I start with image 2 as the mosaic, then warp images 1 and 3 into its frame with their homographies and add them one at a time. Each time, the canvas grows to fit the warped image's bounding box.

To hide the seams, I weight each pixel in the overlap by its distance to the edge of its image (using a distance transform), so each image fades out toward its border. Then I do two-band blending: the low frequencies (Gaussian blur) are blended with the smooth, blurred weights, and the high frequencies use the sharp weights so edges don't ghost. I used σ = 50 for the gateway and σ = 20 for Sather Gate and the tree.

def two_band_blending(im1, im2, mask, sigma):
    low_im1, high_im1 = get_two_band(im1, sigma)
    low_im2, high_im2 = get_two_band(im2, sigma)
    low_mask = cv2.GaussianBlur(mask, (0, 0), sigmaX=sigma)
    mask = mask[:, :, None]
    low_mask = low_mask[:, :, None]

    low_blended = low_mask * low_im1 + (1 - low_mask) * low_im2
    high_blended = mask * high_im1 + (1 - mask) * high_im2
    return low_blended + high_blended

# in add_image_to_mosaic: weight overlapping pixels by distance to each image's boundary
dist1 = cv2.distanceTransform(mask1, cv2.DIST_L2, 3)
dist2 = cv2.distanceTransform(mask2, cv2.DIST_L2, 3)
overlap = (mask1 == 1.0) & (mask2 == 1.0)
blend_mask[overlap] = dist1[overlap] / (dist1[overlap] + dist2[overlap])

The blending hides the exposure differences between shots pretty well. People who moved between shots still ghost (like the walkers under Sather Gate), and the wide-angle gateway shots leave a visible seam on the white chair in front.

Website template from Jon Barron.