CHI paper on the usability of intelligent agents

Last week, half a year of work accumulated in a 6 page paper.

For my Research Project I’ve been working on the usability of intelligent agents. To give you a quick update: my main hypothesis is that there are very capable intelligent agents as well as rather less capable ones, but that usability not only depends on system capabilities, but also on the appearance of the system. Specifically, a capable agent should look “intelligent” so that users immediately understand that they can use a rich interaction, approaching the richness of human-human interaction. Otherwise, users may underestimate the system capabilities, and not use its full functionality. A less capable agent, on the other hand, should not look too “intelligent”, because otherwise users may overestimate its capabilities, and wonder why such a smart-looking system doesn’t understand their commands.

HCI or HTI people will see that I’m drawing a parallel here with Norman’s idea of feedforward (the appearance) and feedback (the actual system response) helping to establish a use image (the inferred intelligence). The trick is that users will use a “human-like” use image, and therefore human-like cues can be used as feedforward. In fact, the more human-like the appearance of the system, the more intelligent the system is believed to be.

I used a trick to prove this hypothesis: I had some systems in which cues and actual system capabilities matched, and some in which they didn’t match. You’ll have to read my paper for detailed results. But one result was very clear: 22% of the participants that used a system with low capabilities and very human-like cues got so confused by the mismatch between feedforward and feedback that they simply quit the experiment after a few minutes!

As I said, I put all this in a 6 page paper, which I submitted to the CHI 2008 student research competition. I will wait putting the paper online until I hear more about that (but you can ask me for it on email if you’re interested). I’ll keep you updated!