PVIP aka Project for Visually Impaired People, one of my most rewarding and challenging project I managed.

Let’s put things in context: Create a solution for blind and visually impaired people to make them able to scan QR codes.

Yes you read well, make blind people being able to find and scan QR codes. It seems a great challenge, and it was, but we somehow succeeded to build a relevant and easy to use application!

Actually the real challenge here was to understand how blind and visually impaired people use their smartphone in Japan, and fortunately my team got support from visually impaired people NPOs and from the Japanese government with their innovation branch NEDO. We conducted research among Japan, delivered early prototypes with limited functions in order to understand more which are the most valuable…

We decided to develop 2 native apps in order to avoid possible performance bottlenecks existing on frameworks. We got a first working version in a month on both platforms, and launched a first big campaign of test all among Japan.

The app was able to scan QR and barcodes very rapidly, provide a simple and accessible interface for VIPs to get into it easily with voice guidance… but it was not enough. How to find a QR code when you can’t see it ? How to place your phone properly to scan it ? What if it takes too long to find one ? How being even able to know there is a QR code ? A lot of questions, and a technical possible solution: Machine Learning.

We then started to develop ML models (only on iOS with MLKit as a pilot), something new to me and very challenging! Try to find a square shaped object with the help of ML… it actually finds that almost all objects as square shaped (look around, your laptop screen, the door of your room, your box of masks… are recognised as square shaped objects). So… we trained the model with a lot of QR codes, defined the app behaviour with the camera in real time, and even implemented object recognition in order to get a simple audio description of what’s on the screen. A bit like the iPhone camera app with VoiceOver activated!

My team was distributed among Japan and Vietnam, so we needed to be great on the communication and the workflow. Fortunately everything went well, and with great preparation, we even delivered better results than expected!

Moreover, this project aimed to help people with disabilities with REAL solution built with them, for them. Just imagine this accomplishment feeling! It was amazing.