CAPITAL CORP. SYDNEY

73 Ocean Street, New South Wales 2000, SYDNEY

Contact Person: Callum S Ansell
E: callum.aus@capital.com
P: (02) 8252 5319

WILD KEY CAPITAL

22 Guild Street, NW8 2UP,
LONDON

Contact Person: Matilda O Dunn
E: matilda.uk@capital.com
P: 070 8652 7276

LECHMERE CAPITAL

Genslerstraße 9, Berlin Schöneberg 10829, BERLIN

Contact Person: Thorsten S Kohl
E: thorsten.bl@capital.com
P: 030 62 91 92

2018 – PVIP

PVIP aka Project for Visually Impaired People, one of my most rewarding and challenging project I managed.

Let’s put things in context: Create a solution for blind and visually impaired people to make them able to scan QR codes.

Yes you read well, make blind people being able to find and scan QR codes. It seems a great challenge, and it was, but we somehow succeeded to build a relevant and easy to use application!

Actually the real challenge here was to understand how blind and visually impaired people use their smartphone in Japan, and fortunately my team got support from visually impaired people NPOs and from the Japanese government with their innovation branch NEDO. We conducted research among Japan, delivered early prototypes with limited functions in order to understand more which are the most valuable…

We decided to develop 2 native apps in order to avoid possible performance bottlenecks existing on frameworks. We got a first working version in a month on both platforms, and launched a first big campaign of test all among Japan.

The app was able to scan QR and barcodes very rapidly, provide a simple and accessible interface for VIPs to get into it easily with voice guidance… but it was not enough. How to find a QR code when you can’t see it ? How to place your phone properly to scan it ? What if it takes too long to find one ? How being even able to know there is a QR code ? A lot of questions, and a technical possible solution: Machine Learning.

We then started to develop ML models (only on iOS with MLKit as a pilot), something new to me and very challenging! Try to find a square shaped object with the help of ML… it actually finds that almost all objects as square shaped (look around, your laptop screen, the door of your room, your box of masks… are recognized as square shaped objects). So… we trained the model with a lot of QR codes, defined the app behavior with the camera in real time, and even implemented object recognition in order to get a simple audio description of what’s on the screen. A bit like the iPhone camera app with VoiceOver activated!

My team was distributed among Japan and Vietnam, so we needed to be great on the communication and the workflow. Fortunately everything went well, and with great preparation, we even delivered better results than expected!

Moreover, this project aimed to help people with disabilities with REAL solution built with them, for them. Just imagine this accomplishment feeling! It was amazing.