Meet Sjoerd: The Technology Behind Touchpulse

Touchpulse is still a young company, but the journey so far has been an exciting one. Earlier this year we launched the beta of our navigation app for visually impaired people, Tiera. After feedback from our community and vigorous testing, Tiera is now out of beta and available on both the iOS App Store and Google Play. As we continue building and improving the technology behind it, we sat down with our co-founder and Chief Technology Officer, Sjoerd van der Goor, to talk about the ideas, challenges and vision driving Touchpulse forward.

Photo of Sjoerd

What inspired you to start Touchpulse and develop Tiera? 

The idea behind Touchpulse originated from a student team I was in at the time, Team HART (Human Augmentation Research & Technology). HART was working on making a wearable sleeve at the time, which would, using vibrations, provide information to the wearer that they could otherwise not receive. We started with a language (specific patterns meaning specific sounds in the English language), so that people who are deaf could ‘hear’ the spoken word of someone they are listening to, but later, the underlying technology was used for conveying facial expressions such as smiling and frowning, so people with a visual impairment could ‘see’ the facial expressions of people around them. Around this time, Liam and I met, and we had interviews with various people about whether the technology had any merit, but visually impaired people (VIPs) are actually able to hear the emotions from someone very well, and the product was very new and quite expensive to manufacture, so it did not work out.

So we pivoted, and asked the same interviewees about their lives, and struggles. We then realised that outdoor navigation was an unsolved problem, which could be solved with novel technologies like AI and Augmented Reality. My inspiration for then continuing with the project-now-company was fourfold:

• Many people have a visual impairment, and I could help all of them. This is still the primary drive behind my work

• My skills would need to improve rapidly to be able to keep up with the demand that would come from developing such a project. Mentally, this stimulates me optimally and fully, which means I learn a lot. This has not changed as the project matured into a company

• At the time, I was getting together with my girlfriend, whose grandmother is almost completely blind. I sat down with her with early prototypes and she was very happy (as grandmas must be, haha), to hear what I was working on, and I think (although she was biased of course) that she actually saw potential as well

• It’s paid work, so I can keep full focus on the company

I try to make work of having interactions with real users each week, so that I can keep seeing and feeling what the work means to them, and therefore to me.

How has your original vision evolved since you first started?

Surprisingly, not much. The way we make money has shifted (in the benefit of the vision), but the core vision of the company has always been to help people with a visual impairment. The way in which we do this has become more clear as we learned more about the problems that VIPs experience, which led to features like Checkpoints, Tele-assistance and the method of interacting with the AI assistant.

At its core, how does Tiera work?

Tiera consists of four layers of support. Each layer transforms different data into guidance or information for the user to rely on to make very good decisions. “Good” here means safe, comfortable, quick, confident, independent. The four layers are:

• Enhanced Navigation: Tiera takes the location of your phone, and together with publicly available data about the surrounding environment generates detailed navigation instructions and provides guidance. The capability of this system are highly configurable, which leaves the user with a lot of power over what information is relevant to them

• Checkpoints: at the start of your route, you may not know which way you have to go. Checkpoints solves this problem by using your camera feed to provide guidance with specific directions (left or right, or 9 o’clock or 3’o clock), and confirmation with vibration and a sound effect when you are facing the right direction. You can enable this feature at any point during the route to re-orient yourself. It can also automatically activate at the start, when you head off the route, and when you are about to reach your destination, if you enable that from the settings. You can also have it automatically activate if you hold your phone vertically (if Tiera is on the screen of the phone).

• AI Assistant: The AI assistant is a single button press away from being able to answer questions about your route. This can include starting a new route, cancelling a route, changing your language settings, and much more. Soon, it will also be able to ‘see’ on your behalf using your camera.

• Tele-assistance: If none of the above work sufficiently, there will be a human available to pick up the phone and guide you through the situation using your camera and microphone. They have access to your route on a map, demographics information, and other context to be able to provide you with rapid information to help you get back on track. Over time, the Teleoperators will teach our AI how to do their job for them, so that in the long term you will be able to rely on the AI assistant fully, and maintain full independence that way.

(Disclaimer: these features are not able to detect all dangers, and cannot properly provide lane-keeping or obstacle avoidance. Users are always responsible for their own safety & must rely on their O&M skills to navigate around safely. We provide all the digital information needed to make confident decisions, but it’s the user’s responsibility to make the final call.)

What’s something you underestimated when building an assistive tech product?

I think the question should be, “What did you not underestimate when building an assistive tech product?”

I underestimated the complexity of tracking how well features are performing, and which features are most important. Keeping focus on the highest priority work, while balancing that with keeping users engaged, and making everyone feel listened to. The technical challenges were expected, but the sheer number of them was not. Also the amount of growth I had to go through to be able to perform the many roles I now have, I did not understand the costs. Work-life balance is still an ongoing challenge.

Furthermore, the amount of attention to detail required to make a feature feel smooth in the hands of the user, and the amount of time that takes, is vastly more than I expected.

What achievement are you most proud of with Tiera? 

I think I feel most pride when talking about the Checkpoints system at the moment. It is truly unique and the way we have integrated it into our system makes it very powerful in the hands of the user. 

What has working in assistive technology and learning from the visually impaired community taught you? 

Mostly how many different people there are. The way people interact with each other, and with Tiera, how people react to things working or not working, and how people respond to future promises versus what is already there.

Also another thing is how much independence big tech companies take away from people. I strongly believe that we must continue giving everyone full access to all the choices they may want to make, so that we never assume someone wants something.

What’s next for Tiera - are there features or developments you’re particularly excited about?

I am excited about AI and Tele-assistance, as they are both still in the early stages of development and still highly open for new ideas. We also onboarded new people in the team to work on this, and I’m very excited to continue working with them.

Where do you see assistive technology heading in the next decade?

Fully personalised AI assistants with full context about the individual. These assistants will be able to answer any questions, use any tool the user already has access to, and do it quicker, with less hassle, and with full control by the user. I think it will completely level the digital playing field between VIPs and fully sighted people.

I really hope, and want to be one of the front-runners in this regard, that big tech does not get there first as they will start taking away choices from the users. I’m afraid it’s a lost race, but it doesn’t mean we can’t try.

If you could go back to day one, what would you tell yourself?

I would tell myself about the features we now have, and what it took to get them developed. I would tell myself about our future plans, and how we could have made things much easier on ourselves by making a few key decisions much earlier on.

I would tell myself about how we manage our product development and app testing, and how our pipelines are set up.

And I would tell myself to never let go of the connection with the user. The empathy I’ve built up, the understanding I now think I have about our users, all of it was so worth everything else.

I would not tell myself about the costs of running a business. The toll it takes mentally, and how it impacts my personal life. I think I would not be ready to hear that. I think that would have stopped me from growing to who I am today.

P.S. I left out some legal/financial tips that would have saved a lot of money and time.

Try out Tiera today, on iOS and Google Play store!