notes
theatrical chat
23 september 2003
I present an idea for a theatrical chat client that stirs aspects of dramatic performance into textual chat. But first, a review of relevant work.
Emoticons have been wildly successful in instant messaging and chat rooms because they for the first time enabled communication along emotional back channels. Part of their beauty is that they convey succinctly an appropriate range of emotions, and their just-in-time use helps to disambiguate and enrich textual communication. Faces can communicate six basic emotions almost universally (Ekman's six: happy, sad, fear, anger, surprise, disgust). Their purposefully abstract quality allows them to convey emotions at-a-glance, more so than more realistic representations.
Graphical avatars have in the past, allowed for a much wider degree of expressivity and personalization than basic emoticons. This is by design, because graphical avatars were thought of as giving an individual a virtual presence; therefore, it's natural to allow a high degree of customization, such as selecting facial features, or comic characters. Customization allows a person to express his/her identity. The expressivity of avatars usually involves animation. The import of animation to chat is that the receiver of a message can respond with more synchronicity to the text of a message than with static emoticons, allowing a higher degree of verisimilitude to real face-to-face chat. The typical trouble with emoticons is that they are sent asynchronously with the actual part of the message being addressed affectively.
But there are also some problems with graphical avatars. They are not as succinct as emoticons in communicating emotions. They are not readable at-a-glance like emoticons, which since entering into our affective vocabulary, seem to have become almost pre-attentive. Graphical avatars are typically too detailed, too complicated, and reading the emotion from them requires calibration to the relative ranges of emotional expressiveness of each avatar character.
In theatrical emoticonic avatar chat, we combine the synchronicity and presence of avatars, with the succinct, standardized, at-a-glance nature of emoticons. To establish synchronicity, chat in EAC is a performance chat, and text is synchronous to typing. Synchronicity of the response is garnered from the actual facial reactions of the chat participants, as recognized by a computer webcam hookup. Because computer vision can only recognize facial expressions if they are exaggerated, and because users are still at option to be deceptive about the facial emotions that are availed, there is the interesting open question about how such a system can make chatting quite theatrical. An important corrollary is how a user would experience a chat differently in light of having performed emotional characatures through chatting.
A graphical sketch followed by a written sketch of Emoticonic Avatar Chat follows:
Textual sketch: Emoticonic Avatar Chat is a synchronous chat client. Every letter typed and deleted is broadcast in real-time. Say that Mary has just posted a message and Tom begins to type. His message will begin a new line on both their clients. If Mary interrupts, her message will begin a new line below Tom.. this corresponds to interruption in a real-life face-to-face chat. Emoticons are used not manually, but automatically. A webcam hook up on both client machines is fed through image processing software to map each user's expression into a standard set of emoticons. The listener's emoticons are mapped to the right side of the blinking cursor of the speaker. This strategic location allows the speaker to jist the live reaction of the listener without having to refocus attention elsewhere on the screen. Complicated avatars or video would not fit in this space to the right of the blinking cursor, whereas the emoticon is a perfectly small and concise graphic that does fit.
Message history is handled by the conventional log of messages, although EAC adds the twist of synchronous display of the typing of the most current message. If the listener interrupts before a post is committed, there appears graphically an act of interruption. The strategically placed emoticon mediates the real live expressions of the message receiver, thus offering more focused and immediate feedback to the message sender.
Emoticonic Avatar Chat seems to be immediately implementable with current technology for facial recognition and chat protocols.