Think aloud evaluation summary

The think aloud evaluation were performed on a smartphones or tablet devices. The prototype is an app made in the program flinto (posted as the first prototype earlier, including the flinto file).

In all cases the user was direct observed in a controlled environment. The reason here was to catch any user-friendliness and expose issues that can be improved for the final version of the prototype. Direct observation is the most convenient since we have limited time and we get very upfront feedback on how the app is used by different users.

Summary 

Except for any annoyance in the login screen (you can only press the login button) most of the test people reacted to the big statistic table in the home screen. The user doesn't understand what it does or why is it there. To understand the purpose of the app, the user needed a short explanation of what the app really does. Since the home screen is the most important part of the app, the buttons seems to need to be more self-explanatory. Also the "Buy Ticket" button is very big and it might mislead the user to do the things and understand the core idea of the app. 

Here is the Home Screen of the first version of the app:

Final Design

This our final design. We decided to rework the design based on feedback, so this is our second high-fidelity prototype iteration.

On top we have wheels that fill out based on how many trips you have left to unlock a ticket. For example the left one is a three day ticket. When you reach that limit you will not pay more than that during a three day period. This will also be reflected on the “Next Travel Cost” label. It will be set to 0 kr for the ticket period. The “Current Ticket” will also be upgraded so you easily can identify which ticket your are currently traveling by. You can also swipe left and right on the circles for additional tickets but the most relevant tickets will be displayed by default.

After doing Think aloud evaluations we recognised that a lot of design problem revolves around numbers not being presented in an easily understandable manner. We also got feedback on graphical elements not being very intuitive. We took this to heart and discussed a lot about how to order information and how create intuitive labels. We continued developing our product and with the modifications applied the image to the right is our current high-fidelity prototype. We believe that we made great improvements but there is always room for additional iterations and fine polishing.

Our target group are sporadic travelers and their problem being that it is sometimes difficult to determine what ticket saves you the most money. Sporadic meaning that they now and then travel less than the amount that justifies buying a monthly card. Our system makes sure you pay the absolute least amount while still being adapted to the SL set pricing points. With our application the travelers choice problem is entirely resolved and presented in a intuitive and appealing package.

Additional features

Login screen that will appear on first setup. It is required in order to connect to Mitt SL so the app can access personal traveling information.

First statistic screen displays travel amount over several months. So you can easily track how much you are traveling and when. The bars indicating how much you payed that month. You can swipe left and right to find the month you are looking for.

Second statistic screen shows on which day you bought a ticket and it also displays which period an unlocked ticket is available for.

You can swipe left and right here as well to show different months.

Ticket details show the user additional information on expenses. So the user can easily keep track of what cost what amount so that there is absolute transparency in every transaction that is made.

Settings will contain things like language options, billing method, etc. It also allows the user to logout and switch account.

Test cases

We made some test cases to simulate the use of our 'pay as you go' payment method, and how it should work given certain situations. We tested three different users during a month, all sporadic travellers that had to gain by the use of our product. The three different users are represented as each worksheet down in the left corner (on Google Spreadsheet).

What is the data representing?
The spreadsheet helps to visualize the use of our product on specific users. The user is given a cost per travel based on the current price for SL credits ('Reskassa') and then given specific travel zones to travel between (SL have different costs for travelling between different areas). The spreadsheet then calculates the difference between if you would have payed ONLY by SL credits, or if you bought a monthly ticket compared to using our product,.

Case 1:
The first user was given the zones A to B, which correlates to 37.5 kr per travel. The user managed to archive a 3-day ticket during the first three days, and then another 24-hour ticket later during the month. With the use of our app and getting these tickets, the user saved 105 kr (if he/she only had bought SL credits), or saved 220 kr (if the user had bougth a monthly ticket).

Case 2:
The second user was given the zones A to A, which means 25 kr per travel. The user managed to archive a 7-day ticket in the first half of the month. The user saved 50 kr (if he/she only had bought SL credits), or saved 190 kr (if the user had bougth a monthly ticket).

Case 3:
The third user was also given the zones A to A, which means 25 kr per travel. Here we tried to demonstrate how the app continuously is checking backwards for possible new tickets. Unlike our previous examples, this user started traveling in the middle of the month. He/she managed to archieve a 3-day ticket before the end of november, but continued to travel during the start of december, and therefore reaching the cost limit for a 7-day ticket. The user saved a total of 40 kr during november and 135 kr during december (if he/she only had bought SL credits). Or saved 35 kr during november and 55 kr during december(if the user had bougth a monthly ticket).

https://docs.google.com/spreadsheets/d/1sWlAnjZAKxomG9BQveli0L42RZ3eqGnM41Bv1l7qGKs/edit?usp=sharing

Meeting 12/11

During today's meeting we mostly discussed the usability goals of our prototype, what do we actually wanna include in the prototype of our application? Some of us felt as though the application in itself was a bit cluttered, and so we opted to refine our previous designs and our established functional requirements.

What we want if for people to use the "pay as you go fare" alternative. We want the users of our application to feel satisfied knowing they're saving money. The more you travel, the lower the cost, and if you travel sporadically, you won't have to worry about overpaying by buying the wrong kind of ticket. The user experience should be simple. Therefor, our new application will only include one of the three options previously presented, a payment system charging you at the end of each month, either through direct debit or invoice.

Another functional requirement we've opted for is to redesign the stats presented in the application, also in order to better the user experience. Instead of including ticket recommendations, this part of the application will include statistics of your travels. How much does each travel cost? Am I currently traveling for free? How much will I be charged if I keep traveling in the same pattern? Questions like these will all be answered.

The basis of our payment system is the SL Access Credit fee. Every time you travel, this amount will be registered to your personal account. Let's say you travel two times on the first of december, two times on the second of december and four times on the third of december. During each travel, a fee corresponding to the SL Access Credit amount will be charged to your account. Consequently, these three days of travels all add up to a total of 300 SEK, but by using our application our service will recognize that it would be cheaper for you to only be charged the amount of a 72 Hour Card, costing only 230 SEK. Since you didn't know you'd be traveling a total of eight times in this short amount of time, there would have been no way of you knowing that buying a 72 Hour Card, instead of paying for each trip separately, would have saved you 70 SEK. But My SL does.

Other than changing the name of our application to "My SL", no more changes have been made.

Think Aloud Evaluation

The direct observation was performed in a controlled environment on a person well accustomed to the usage of smartphones. During the evaluation, every action was spoken aloud, as they should be.

The evaluation began with me asking the person to explore the application freely. Given only these instructions the person signed in by simply tapping the login-button, no questions asked. I conclude that this particular screen is very intuitive. 

Having the home-screen at display, the person commented on the information present. The comments were mostly about the statistics, saying that the histogram needed numbers to convey the information more clearly and that the green bar needed something along the same lines in order to serve its purpose. In regards to the ”Autogiro” headline, the person thought it would be better to rename it and use the name ”To Pay”. All of this information implies that this screen must be redesigned.

Next, the person, by accident, opened the statistics-screen, being surprised when doing so. Clearly, we have to make it visibly known that the stats presented on the home-screen are clickable. The person thought that this screen was pretty understandable though, only leaving comments on the maximum/minimum cost headlines. ”Does ”Maximum cost” mean that I only can be charged, at the most, 100 kr a day?”

The person found the last two screens to be intuitive.

Following the evaluation, I conducted a one-question-interview, asking about the overall impression of our application. In short, the answer was that the application seemed a bit messy, it was including a lot, maybe a little too much.

I agree. Our application include more than is necessary. If a feature isn’t crucial, it should be discarded. Simple and clean is the way to go.

Final prototype sketches

These are some sketches made for the final prototype. Here we test how our data should be presented to the user. We chose circles for their ability to present data in a rather small area of the screen.

Meeting + exercise 4/11

After our meeting previous day, we divided the work so that a few in the group started to work on the layout for our mobile application the same evening. The images that were created and made into a prototype by using Flinto are presented in a video in a previous blog post (http://group-d2.blogspot.se/2015/11/first-prototype.html).

On this meeting we mainly tested the prototype and had long discussions about the design choices and new ideas about how the layout could be developed. We all liked the outcome, but we thought that maybe there were a little bit too much information and statistics on the main page. We also formed a power point for our presentation and prepared for what we were going to say. Here are our slides:

The feedback we got from the presentation during the exercise was really good! Some of the reactions was:

- Clearifying questions about the main idea and the features
- Some of these features already exist in SL's applications
- Maybe we could incorporate our ideas into being features of one of SL's applications, it might not be neccessary to create an entirely new application.

Think-aloud evaluation, 4/11

I did my think-aloud evaluation by asking a friend who had never heard about our project before, nor seen anything about what we have been working on, to look at the prototype application and tell me everything he thought about it.

Apart from some difficulties trying to understand how the touch worked and how to get around on the application (where to push), the first reaction I got was “wow, there are lots of digits!”.

The participant is a young guy who is used to travel by the public transportation in Stockholm, and also comfortable with using mobile applications. After looking at the prototype for a few seconds he seemed a bit confused, and so I told him briefly about the main idea and the purpose of the application.

After being told that the app was meant to support buying tickets by autogiro, give you a recommended ticket and show statistics, the reaction was “where is my recommended ticket?” and he also commented that it was hard to get a grip on the statistics, it contained too much information.

Overall, he seemed to like our concept, especially since he only travels occasionally and really would benefit of paying only for the travels he makes, without having to worry about which ticket to buy. Usually though, he wouldn’t want to buy a monthly ticket based on a recommendation, until he knows how much he is going to travel the upcoming period, which can make some of the statistics somewhat redundant. Either way, he liked the tag showing the price per ticket, and thought that we should develop the idea to make it more easy to read.

First prototype

We built the first prototype in the recommended program Flinto (desktop version), which was not very clever. The desktop version can only export the file with .flinto extension, which means we can't share the prototype in the Flinto online version (it doesn't import). To view our prototype you can either watch the screen cast, or download the desktop/iOS/Android version of Flinto.

Here is the link to the Flinto file (since this is a course-recommended application we assume that the examiner can use this file):
https://drive.google.com/file/d/0B_8uNNylE9zSWEQtSVZWajZjWWM/view?usp=sharing

The screen cast is doing the following steps:

1. Tapping login
2. Tapping on the statistics
3. Tapping in the same place again to get back
4. Tapping "Buy Ticket"
5. Tapping the blue top border
6. Tapping the white settings icon (with gray background)
7. Tapping Logout

Note that I press beside the tapping area to get the app to show with a blue flash which areas that are interactive (this is a Flinto prototype feature, not intended to be a feature in our app).

This is the very first high-fidelity prototype that among others will clarify vague requirements, think aloud evaluation with a user and check that the app just seems to be in the right direction to solve our problem. Since we don't write any code ourselves (which we actually are more than capable to do) this would not be considered a very expensive thing to do, even though in general the high-fidelity prototype would be.

Here we did have to compromise a lot with the features to get the app to seem easy to use. It got very clear that some features that we though would be needed in this app actually did not solve the key issues that we had. This was easier to understand when we were prototyping this first interactive version of the app. It also got very clear when we thought about what functions the will perform, the app should get people to save money and make the payment easier.

Think Aloud evaluation

My participant is someone who travels a lot with SL and is also very comfortable with smart unit interfaces.

I gave him the task to explore the application and meanwhile voice his impressions and what he was thinking at each stage.

The person is immediately meet by the login screen

<try frantically tapping user-name field and forgot-password field>

Comments that buttons are not working

<tries pushing login>

<looks around for abit>

<push on the statistics>

He comments on the numbers not making any sense.

Says that it does not make any sense that 2 journeys cost 425 kr. (Which is correct but they are only placeholders)

Also comments on the green bar, asking if it is how many days that are left on this month.

<tap return button>

<tap buy ticket>

<tries tapping different buttons seemingly at random>

Asks if any of them are working yet. (No)

<tries swiping left>

Comments that you should be able to swipe to return

Says that the return button is too small.

<taps return button, immediately taps settings button>

<tries both settings button then logout>

His main points of concern was about lack of explanation on the statistics part. He said that it could be improved a lot in order to be more useful.

Think aloud evaluation

Think aloud evaluation 

The person that testet the first version of the app is a student at KTH, which implies that he is most likely well used to smartphones and modern apps. 

This is performed on an iPhone 6 plus during a direct observation in a controlled environment by myself standing next to the test person. The test followed with a short interview to explore issues as well as capturing the detail of what the user do.

The person is presented with the login screen and successfully logs in with his My SL account. <tapping the login button>

When the home screen appears he tries to scroll down the list with the statistics. He swipes multiple times in all direction, double taps, until (after almost 40s) he taps one time and the statistic page expands. <tapping the statistic tables>

He taps again and gets back to the main screen. 

Without any delay and he gets in to buying tickets. <pressing Buy Ticket-button>

He taps everything, which is not yet clickable and immediately finds his way back to the home screen.

The last part is settings and he finds the button from the home screen. <pressing settings button>

Here he managed to log out.

The test person's final thoughts were much about the big statistic panel in the middle of the home screen. He said that he expected the interaction to behave different since the panel did not expand by swiping.  

Think aloud Evaluation

The user was given the task to find their way around our Hi Fi-prototype in Flinto, in a controlled environment and then give feedback on the design and functionality of the service. This was the feedback received:

Nice overall design, the login button could be better with a darker blue color. The button for ‘Buy Ticket’ is quite big and distracting. The statistics are very unclear and confusing, but a nice feature to have if made more interesting. The card balance and monthly cost tab on the home screen is clear and simple.


In conclusion from this session, we need to make the prototype:

• More interactable
• We need more slides in Flinto for changing the settings in 'My SL', change language, more information about our new payment service and maybe the ability to change the current viewed access card.
• More explanatory
• A lot of the statistic slide need to be more explanatory with more labels and descriptions of what the graphs mean. Also we need more information about our tickets.
• More accessable
• We need to rethink our design of our main page to prioritize the needs of our user. The 'Buy ticket' button is maybe too big and distracts from the overview we strive to archieve. Also the statistics might be too excessive at the start, and we probably need to rethink how to present that data to the user in a self-explanatory and still good looking way without too much information or clutter on the screen.

Think-aloud evaluation

Think-aloud evaluation was done with letting a user test the prototype while talking, while taking notes.

What seemed to work well:

The design looked good, while there still being some quirks to sort out in for example the colour-palette.

Kind of easy to use

What seemed to work not as good:

The buttons are not self-explanatory enough to show what the user is doing.

The prototype doesn’t have all the buttons and links designed yet, so you can’t for example “login” with anything nor press a couple of buttons.

The target group is well-defined but yet very broad and thinking logically about who might use this service will give the conclusion that every commuter with a smartphone can benefit from having this application.

Since this course is focused on HCI and making platforms user friendly, it’s quite clear that the design matters just as much as the general idea. It’s very important to design a good user interface and make the whole application easy to use and understand without having redundant information or too much text. To make this process easier, you’d need to do several more think-aloud evaluations.

First one on the first prototype will reveal big flaws. When you later on have everythin sorted out and feel pleased with the design, you’d need to do a new evaluation to let someone “outside” the project see things that you might’ve not seen. Testing over and over will in the end give the best result for this specific project.

The weak points will from this iterative testing be revealed over and over until we have a very solid foundation to start making adjustments to the design for other "hooks" in the application. If you'd for instance want to add a map, you'd need to redesign alot of the application to make it seamless and therefore do even more testing. Therefore, a good solution to this is to have wiggle room in your project to avoid "dead ends".

Prototype sketch

These are our sketches we made before moving into flinto. The paper prototypes purpose is to set a concrete design and ensure we have the functionality that we want.

Meeting 3/11/15

Today we discussed the direction of our project based on the feedback from the prevoius exercise and our own reflections. It seems that we had a slightly different vision about the features our product offers and we needed to go back to our main problem and discuss what our product had to solve, and how our current product and features solve this. We stated that our main problem as described in our personas where to save money and travel conveniently. We then broke apart the content of our current proposal and discussed how each feature helps to solve this main problem.

The issue discussed the most was the direct debit payment method presented in our last proposal. Some in the group argumented that this feature was not realistic to archive due to the 'loss in income' SL would have from the result of travellers saving money. Something SL most likely never whould have approved. But after a discussion and some backround research done earlier in the project about a similar payment method in London (http://group-d2.blogspot.se/2015/09/contactless-payment-in-london-metro.html) we had a vote and decided that we wanted to implement this payment method, and assume that SL whould set balanced prices to cover up possible losses.

We also discussed how the statistics could be helpful for the traveller and how it could be used to solve our main problem. One assumption that we have made for the possibility of this feature to exist, is that we must have access to SL's statistical database in order to extract relevant travelling data and present to our user. We do not know if SL have this type of information available, but with the size of SL in mind it would be very surprising if they didn't.

We then decided to start the design work of our protoype and listed all our main pages and functions which later will be assembled into a working interactive prototype using Flinto. We listed the following menues and content:

First login:

• Login
• Connected to 'Mitt SL' (My SL)
• Possibility to create an account
• (If you don't have a connected access-card to your account) Connect a card

Content:

• Homescreen
• Fast balance
• How much money left on the card
• How many days left of your card
• Statistics
• Statistics for current activated cards
• Statistics for recommended tickets based on previous travels
• Tickets
• Possibility to buy tickets from the app
• Possibility to subscribe to a ticket
• Gives ticket recommendations based on statistics from previous travels
• Possibility to pay by direct debit during a certain period of time, where the price becomes cheaper the more you travel (When you reach a certain price limit)
• Settings
• Change language
• 'Mitt SL' (My SL)
• Connect an access-card
• Connect a bank card
• Connect a Mecenat card (For students)
• Report a stolen access-card
• Change account details
• Log out

Here are some sketches of our first prototype.