During the work with projects and products, evaluations should be done of design functions. Depending on the goal of the evaluation, different methods can be used. The DECIDE framework is using six statements with focus on the goal, identifying the issues and thereby choosing the method and the relevant questions, and how to analyze the data. It also talks about dealing with ethical issues, such as participants’ privacy, and how it is important to keep for example personal records separated from the collected data in a safe way. By reading the report, you should not be able to identify the participants, if they haven’t given their permission. Before the test, a consent form is useful, so that the participant know all about their involvement and their rights.
By going back and forth between the statements, they are supposed to help you to choose the best ways to perform your evaluation process. Before starting the process it is important to think about all aspects, as well as who the participants should be, or if you have the right equipment, resources, knowledge or enough time to perform the chosen method.
When analysing the data, it is important to think about the reliability - how reliable the result is, the validity - if the data meet the goal of the evaluation. The ecological validity is a measures of how much the used environment has affected the result. The scope of the study says how widely the information can be used, or if the results are really narrow.
Chapter 15 gives examples of different evaluation methods where no users are needed. These kinds of models are called inspection methods. One of them is the heuristic evaluation, which is a method of testing user-interfaces, by using experts instead of actual users, with focus on how accessible the product is, e.g. if it speaks the user's language, how easy it is to use,if the language is consistent and if the purpose of the use can be fulfilled.
Walkthroughs is another method where the purpose is to predict the user problems. During cognitive walkthroughs, the participant can be told to do a task during a given scenario, while documenting the process and the occurring problems. During pluralistic walkthroughs, several experts try out a scenario together. They do the task separately, and does then discuss their different solutions to solving it.
Analystics is a method were user traffic on a system is being evaluated, which is often used by presenting statistical data. One dilemma is that the users not always are being aware that they are being “watched”. The predictive model is similar, but doesn’t involve users. Instead predictions are made based on role-plays and estimations. Two well used predictive methods is the GOMS model and KLM model.
Queston: What are the factors that determines the scope of an evaluation?
No comments:
Post a Comment