UNet for Paper Lightbox Stencil

I was making paper lightboxes for fun at home and I had come across a large set of free templates online. I noticed that the templates were png or jpeg format and they all contained dotted lines. As I typically use my laser cutter to cut the paper out, I needed SVG format stencils. Typically it isn't to difficult to use Ilustrattor to trace the vectors of fully connected lines, however, due to the lines being dotted, this was not possible.

I had recently learned about U-nets and how they can utilize convolutions to encode local features while using skip or jump connections to maintain global context of the image. While I had used convolutional neural networks before, I had never implemented a U-net before and I thought this architecture would be useful as the encoder might be able to identify the gaps that needed to be filled, while the skip connections could maintain the overall structure of the line.

Project Goals

Create a dataset of images that contain dotted lines and the corresponding solid line images.
Write a UNet model architecture that input 1024x1024 images and output 1024x1024 images.
Train model on dataset and evaluate performance.
Utilize Adobe Illustrator to convert the output images to SVG format.
Cut paper light box from SVG file.

Tech Stack

Image Processing: PIL (Python Imaging Library), OpenCV, Numpy
Machine Learning: PyTorch

Example

Input Image:

Output Image:

Target:

Challenges

Managing Empty Space in Training Images: Training images contained significant amounts of empty space, leading to longer training times and reduced efficiency in model learning.
Illustrator Limitations: Due to limitations in the image tracing algorithm in Illustrator, the lines were not always traced correctly, leading to inaccuracies in the final SVG files.

Lessons Learned

This project was a great learning experience in both image processing and machine learning. I learned how to preprocess images for training, design and implement a U-Net architecture, and manage training data efficiently. Overall, this project helped me improve my skills in both image processing and machine learning and provided valuable insights into the challenges and complexities of working with real-world data.

Future Improvements

Create a more diverse dataset by generating images that are full of lines to improve model generalization.
Modify model architecture to encode lines directly to SVG format. See this.