Status quo

With conversational interfaces, spoken or written language is used to execute tasks instead of clicking several buttons on a graphical interface. However, there are often inconsistencies between users’ expectations regarding the capabilities of CUIs and their real skills, leading to frustration and disappointment. Taking into account the current trend of applying CUIs, users will presumably encounter them more frequently. Yet, it seems like there is room for improvement concerning the design of these interfaces to provide a successful conversational user experience. These findings lead to the following research question for my thesis:

How might we enhance both user acceptance and user experience of conversational user interfaces?

Research

I undertook semi-structured interviews with six users to identify and understand their pain-points and needs concerning their previous interaction with conversational interfaces. Based on the gathered findings, I chose a potential use case for CUIs to sharpen the focus of my thesis.

The interviews indicated that users currently use CUIs for rather simple tasks as they cannot imagine conversational interfaces for more complex use cases yet. Still, the discussions about various use cases of chatbots have shown that a general interest in CUIs and their new areas of application exists. This became clear when existing solutions are complex or when using a conversational approach alleviates and supports users.

I also gathered the participants’ opinions about current applications of chatbots and what desires and rejections they have about 15 potential use cases for conversational user interfaces. For each topic, they were asked to rate their willingness to use such conversational agents from 1 (very unlikely) to 7 (very likely) and explain their decision in more detail.

I focus on trust to keep things concise. Respondents reported low trust in CUIs' competence and doubted the feasibility and convenience of chatbots. Their confidence is limited due to perceived risks and concerns about security, especially with sensitive data. Surprisingly, they do trust task-oriented dialog systems, valuing their usefulness and ease of use.

Next, I selected the use scenario for my thesis. Despite financial service chatbots scoring an average of 3.33, trust issues were frequently mentioned. From a UX perspective, this lack of trust is particularly intriguing and worth exploring.

What is conversational banking?

Conversational banking, the latest in digital banking, enables customers to use natural language via social media, messaging apps, and voice assistants to access financial services. This offers financial institutions a cost-effective way to deliver personalized digital experiences.

Younger generations expect personalized, flexible banking solutions similar to the apps they regularly use, and older customers are also dissatisfied with current mobile banking options. Fintech companies add pressure by offering innovative, user-friendly IT solutions that surpass traditional banking services. Conversational interfaces in mobile banking can enhance user experience and help banks address both technological and socio-economic challenges.

Ideation

I implemented three tasks for all CUIs to ensure data comparability, based on common banking app functions with varying complexity:

Next, I outlined the conversation flow for the three mobile banking use cases and visualized it to assess the task flow and identify terms users would naturally use.

I tested the proposed dialog and use flow with users. Their feedback focused on the assistant's wording, which caused ambiguities and the conversational flow. Previously, the assistant asked for the recipient and amount simultaneously for invoice transfers, but users often responded to only one. This feedback was incorporated into the final prototypes.

After refining the task flow, I adjusted the conversation design:

Prototype

To address user security concerns and ensure realistic prototypes, I incorporated PSD2 policy elements, including Strong Customer Authentication (SCA). This requires two-factor authentication (2FA) for online payments, using at least two of the following:

• Knowledge: Something the user knows (e.g. password or PIN)
• Ownership: Something the user owns (e.g. smartphone or smartcard)
• Inherence: Something unique to the user (e.g. fingerprint or face recognition)

General features

All CUIs handle the three use cases consistently, with NEO always acting the same way. While users control the sending or transferring of money, the interface implementation varies. Thus user control, highlighted as essential in the preliminary study, is maintained. Additionally, all CUIs provide feedback on the success of actions with relevant details.

Speech-based CUI

After login, the CUI introduces its capabilities and the virtual assistant through a welcome screen. The speech version allows users to control input by starting and stopping recordings. To prevent errors, users can always start a new recording, which deletes the previous one. Sensitive data, like account balances or transactions, is displayed visually, not read aloud, addressing concerns about social embarrassment and privacy from the preliminary study.

Hybrid CUI

After login, users are taken to the home menu with buttons linking to various functions, each with a brief description. If the wrong function is selected, users can return to the main menu with the back button. This ensures easy access to the assistant's capabilities and allows users to restart or end interactions. A fixed UI element displays relevant details in the chat, improving context and system transparency.

Text-based CUI

The two prototypes were compared to this text-based chatbot version, often used on messaging platforms. Before transferring an amount, regardless if a TAN is required or not, the agent shows payment details and asks for confirmation. Users then initiate the payment with a text-based confirmation.

The following video shows the implemented use cases with the three different prototypes in action:

User Study

18 participants tested the three prototypes in a randomized task order, completing the same tasks with different CUIs to reduce carry-over effects. The study included several context-adapted questionnaires: technology acceptance (Davis), trust (Jian, Bisantz, and Drury), UEQ-s (Schrepp, Hinderks, and Thomaschewski), and perceived security (Cheng, Lam, and Yeung). Participants ranked the prototypes and participated in short interviews.

Based on the literature and prototypes, I formulated the following hypotheses:

• H1: Acceptance differs significantly between the prototypes
• H2: Trust differs significantly between the prototypes
• H3: User experience differs significantly between the prototypes

Results

The statistical evaluation found no significant differences between the prototypes, so the hypotheses were not supported. However, design implications for conversational banking interfaces can still be drawn from the data.

Acceptance:
In the interviews, 11 of 18 participants expressed interest in using conversational interfaces for financial services, noting they were "easy to use", "supportive", "responsive to personal needs" and "clearer" than current solutions. Five were open to using it as an add-on to existing apps, while two rejected the idea.

Trust:
The speech-based CUI scored higher than the hybrid one, despite initial concerns. Trust is complex and influenced by factors beyond design, like security. Users valued security more than speed, suggesting clear communication of security prompts could build trust.

User experience:
The voice-based CUI scored highest for both pragmatic and hedonic quality. Despite concerns about voice input and errors, many users found it "efficient", "pleasant", and "easy to use", indicating that its benefits outweighed the drawbacks.

Perceived security:
Perceived security was more influenced by the use case and security prompts than the interface design. A touch-ID prompt, for example, increased users' sense of security, though the most favored interface wasn't always seen as the most secure.

Conclusion

In summary, users have a positive attitude toward conversational interfaces for financial services, with a preference for the hybrid approach. Key considerations for improving user acceptance include:

• 01 Support modality
  Users want the option to interact via both voice and text, allowing flexibility depending on their environment.
• 02 Enhance transparecy
  Clearer feedback is needed to clarify processes, such as confirming whether payments have been successfully processed.
• 03 Provide shortcuts:
  Text-based interfaces can feel tedious, so adding predefined buttons can make interactions faster and more efficient.

Additional features like invoice scanning, reminders for due invoices and receipt notifications were also suggested.

Learnings