Capstone: Braille Haptic Reader
Custom translation accuracy under varied lighting
Full image-to-haptic-pattern latency
Python scripts, YOLOv5 integration, solenoid driver loops
Raspberry Pi 4, high-res camera, 3D-printed haptic cell array
Deep tech integration pairing computer vision & physical solenoids
About the Project
As my capstone project, I developed an OCR-to-Braille translation pipeline running on a Raspberry Pi. The system uses a camera to capture text, which is then processed by a custom-trained YOLOv5-based AI model for accurate text detection. The detected text is run through an OCR engine and translated into Braille, which is then outputted to a haptic display. The system achieved a 97.82% OCR accuracy, enabling real-time translation for visually impaired users.
Challenges
Training a highly accurate text detection model for various fonts and lighting conditions was a significant challenge. Integrating the entire pipeline—camera capture, AI inference, OCR, and haptic output—on a resource-constrained device like the Raspberry Pi required extensive optimization. The real-time constraint meant every part of the process had to be as efficient as possible.
Learnings
This project taught me a great deal about the practical application of machine learning models on edge devices. I learned about optimizing deep learning models for performance, working with hardware interfaces, and building a complete, end-to-end system that serves a real-world purpose.
Development Journey
Hardware Prototyping & Pin Mapping
Sourced Raspberry Pi 4 board, configured custom-designed circuit connections, and soldered 3D-printed haptic cell solenoid relays.
YOLOv5 Character Box Detections
Trained high-accuracy character and paragraph boundary detection anchors with YOLOv5 on various distinct printed fonts.
Python Braille Mapping Core Engine
Developed low-level Python scripts converting OCR string characters into 6-dot haptic grid arrays mapped directly to GPIO pin nodes.
Thread Tuning & Validation testing
Streamlined image processing threads to cut pipeline lag to ~1.2 seconds, achieving solid 97.82% overall accuracy.