Finishing my PhD and beginning a new project

Having carried out the final experiment of my PhD research, all that’s left to be done is to wrap up the past four years in a nice thesis. The main body of the book is already present, the structure needs some careful thought; the main things I have to create are the introduction and the conclusion. Release date: this year. Title reveal: soonish.

I’m also happy to say that I’ll be staying at HMI, yet working on a different project from now on: R3D3, the Rolling Receptionist Robot with Double Dutch Discourse. Let me quote the project plan on this:

The R3D3 project aims at investigating situated natural language dialogue systems that combine limited natural language understanding with the understanding of the nonverbal behaviour of the users in a real-life context. Inspired by Star Wars’ R2D2, we propose to develop an even more advanced robot, our R3D3. R3D3 takes the form of a Receptionist Robot that can have an intelligent dialogue with the visitors to a museum, shop or office building. These dialogues are ‘double’ in the sense that they involve a third party in the form of an avatar or a person shown on a screen by the robot. The R3D3 project addresses scientific ICT challenges in the areas of computer vision, spoken language interaction, and data processing, as well as practical challenges in robot development.

The mock-up below gives an overview of the capabilities of R3D3.

r3d3

My main responsibility is structuring the interactions of the robot, taking care of the dialogues with both the robot and the virtual agent. My background in interactive storytelling and virtual agents will surely prove its worth to do so. I’m on board with several companies and institutes to investigate how such a robot could be used in several situations. VicarVision will focus on the vision systems of the robot, incorporating their FaceReader technology. The design and construction of the robot will be carried out by Edwin Dertien from the department of Robotics and Mechatronics at the University of Twente. Sentient will incorporate data analysis techniques in R3D3 for end users. Together, we will look at how to place R3D3 in different venues: the Nemo Science Center, the Dutch Police Academy, and in actual shops, thanks to iretail.solutions.