Pushkar Shukla MS Project Defense

Tuesday, June 25, 2019 - 11:00am to 12:00pm
HFH 1132
What Should I Ask? From Asking Visually Relevant Questions to Goal-oriented Visual Dialogue and Beyond
Pushkar Shukla
Matthew Turk (Co-chair), William Wang (Co-chair), Tobias Hollerer


Asking questions to achieve a goal is an important human ability, as it allows us to gain knowledge, reduce uncertainty, and perform tasks more efficiently. Likewise, an artificial agent can be much more useful if it is able to increase its knowledge by asking relevant and engaging questions. The focus of the talk will be on visual question generation, i.e. generating questions with respect to an image. During the first part of my talk, I will be focusing on building deep learning models to generate natural questions with respect to an image. The next part will focus on understanding how humans tend to select a question from a set of questions with respect to a goal and leveraging mathematical models for human inquiry in order to build better systems for goal oriented visual dialogue. Finally, I will discuss a relevant application that involves a guide trying to navigate a tourist with visual feedback from one point to another using natural language conversations.

Everyone welcome!