13 pages

A Knowledge-Based Software Life-Cycle Framework for the Incorporation of Multicriteria Analysis in Intelligent User Interfaces

of 13
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
A Knowledge-Based Software Life-Cycle Framework for the Incorporation of Multicriteria Analysis in Intelligent User Interfaces
  KABASSI, K. & VIRVOU, M. (2006). A KNOWLEDGE-BASED SOFTWARE LIFE-CYCLE FRAMEWORK FOR THE INCORPORATION OF MULTI-CRITERIA ANAL-YSIS IN INTELLIGENT USER INTERFACES. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 18(9), PP. 1-13 1 A Knowledge-based Software Life-CycleFramework for the Incorporation of Multi-Criteria Analysis in Intelligent User Interfaces Katerina Kabassi, Maria Virvou   Abstract  —Decision making theories aiming at solving decision problems that involve multiple criteria have often been incorporatedin knowledge-based systems for the improvement of these systems’ reasoning process. However, multi-criteria analysis has notbeen used adequately in intelligent user interfaces, even though user-computer interaction is, by nature, multi-criteria-based. Theactual process of incorporating multi-criteria analysis into an intelligent user interface is neither clearly defined nor adequatelydescribed in the literature. It involves many experimental studies throughout the software life-cycle. Moreover, each multi-criteriadecision making theory requires different kinds of experiments for the criteria to be determined and then for the proper respectiveweight of each criterion to be specified. In our research, we address the complex issue of developing intelligent user interfaces thatare based on multi-criteria decision making theories. In particular, we present and discuss a software life-cycle framework that isappropriate for the development of such user interfaces. The life-cycle framework is called MBIUI. Given the fact, that very little hasbeen reported in the literature about the required experimental studies, their participants and the appropriate life-cycle phase duringwhich the experimental studies should take place, MBIUI provides useful insight for future developments of intelligent user interfacesthat incorporate multi-criteria theories. One significant advantage of MBIUI is that it provides a unifying life-cycle framework that maybe used for the application of many different multi-criteria decision making theories. In the paper, we discuss the incorporationfeatures of four distinct multi-criteria theories: TOPSIS, SAW, MAUT and DEA, give detailed specifications of the experiments thatshould take place and reveal their similarities and differences with respect to the theories. ——————————      —————————— 1 I NTRODUCTION HE use of computers has rendered the performance ofmany tasks more efficient than it used to be. However,it has also introduced new kinds of problem which aremainly due to the complexity of user interfaces. Such prob-lems are addressed by Intelligent User Interfaces (IUIs) thataim at improving human computer interaction.Common approaches proposed in the literature for in-corporating intelligence in user interfaces include probabil-istic reasoning through Bayesian Networks, machine-learning algorithms, such as neural networks and Case-Based Reasoning. All of these techniques try to model theuser’s reasoning process and have proved to be rather effec-tive. However, a user interface that provides intelligentadvice should also be able to reproduce human advisors’reasoning. For this purpose, decision making theories seemvery promising. Indeed, decision making theories havebeen used for selecting the best information source when auser submits a query [1], modelling user preferences in re-commender systems [2], selecting the best route in mobileguides [3] or individualising e-commerce web pages [4], [5].The decision making theories that seem to be more ap-propriate for computer problems are the multi-criteria ones.This is due to the fact that computer problems usually in-volve several objectives and criteria. Decision making theo-ries provide precise mathematical methods for combiningcriteria in order to make decisions. However, they do notdefine the criteria. Therefore, the first step for applying anydecision making theory is to conduct an empirical study forselecting the criteria that are usually taken into account by ahuman decision maker. Furthermore, many decision mak-ing theories require other additional empirical studies forapplying the mathematical model proposed. For example,additional empirical studies may be needed for the estima-tion of the weights of the criteria etc. Thus it is evident thatthe adaptation and application of decision making theoriesinto intelligent software, requires the conduction of empiri-cal studies at the early stages of the development process.Consequently, the design of these experiments affects theefficiency of the resulting IUIs. Indeed if the experimentsare not carefully designed and implemented, then there is apossibility that useful pieces of knowledge are missed outand the application of the decision making theory fails inthe end.Despite the importance of the development process ofIUIs that are based on multi-criteria theories, very little in-formation is reported in the relevant literature about it. As amatter of fact this is a problem that concerns the IUI litera-ture in general. Delisle and Moulin [6], after an exhaustivereview of the relevant literature, have come to the conclu-sion that there is a shortage of guidelines available for thedevelopment of IUI applications. In the specific case of IUIsthat involve multi-criteria theories, there are many devel-opment steps that are required for their effective applica-tion. These steps are neither trivial nor adequately de-scribed in the relevant literature. This is probably one im- ————————————————   •   K. Kabassi is with the Technological Educational Institute of the IonianIslands, 2 Kalvou Sq., 29100 Zakynthos and the University of Piraeus, 80Karaoli & Dimitriou St., 18534 Piraeus. E-mail: kkabassi@unipi.gr.   •    M. Virvou is with the University of Piraeus, 80 Karaoli & Dimitriou St.,18534 Piraeus. E-mail: mvirvou@unipi.gr.   T  2 portant reason why decision making theories have not beenwidely used in IUIs, despite the fact that their applicationseems very promising. Thus, it is quite important to defineand present a life-cycle framework for the incorporation ofa multi-criteria theory in an IUI. Given the fact that thereare many multi-criteria theories it is also important to high-light similarities and differences of these theories in termsof the process of their possible incorporation into IUIs.Therefore, the main focus of the present paper is on how adecision making theory can be incorporated in an IUI.In view of the above, our research has focused on creat-ing a generic software life-cycle framework of how a deci-sion making theory can be applied effectively in an intelli-gent user interface. The resulting framework is calledMBIUI (Multi-criteria Based Intelligent User Interface) life-cycle framework and involves the description of a softwarelife-cycle that gives detailed information and guidelinesabout the experiments that need to be conducted, the de-sign of the software, the selection of the right decision mak-ing theory and the evaluation of the IUI that incorporates adecision making theory.As an example of use of the MBIUI framework, we havedeveloped MBIFM, which is an IUI that bases its reasoningon the decision making theory called TOPSIS (Techniquefor Order Preference by Similarity to Ideal Solution) [7].However, in this paper, in the context of the MBIUI frame-work we discuss issues about possible application of othermulti-criteria theories such as SAW (Simple AdditiveWeighting) [7], [8], MAUT (Multi Attribute Utility Theory)[9] and DEA (Data Envelopment Analysis) [10].The main body of the paper is organised as follows: Sec-tion 2 describes related work in software life-cycleprocesses and IUIs. Moreover, we give a very brief descrip-tion of the four decision making theories that are discussedin the paper. Section 3 presents an overview of MBIUIframework. MBIUI is further analysed in the subsequentsections, where we give examples of its use for the devel-opment of MBIFM using TOPSIS. 2 R ELATED W ORK   2.1 Software life-cycle processes As systems become more complex, their development andmaintenance is becoming a major challenge [11]. This isparticularly the case for software that incorporates intelli-gence. Indeed, intelligent systems are quite complex andthey have to be developed based on software engineeringapproaches that are quite generic and do not specialise onthe particular difficulties of the intelligent approach that isto be used.This problem has given rise to research that is orientedtowards bridging the gaps between software engineeringapproaches and the development of special purpose intelli-gent systems. For example, Del Socorro Bernardos [12] pro-poses a general framework that helps develop a naturallanguage generation (NLG) project from the conception ofthe need to the retirement of the product. This frameworkis based on the IEEE standards 1074-1997 [13]. ADELFE[14] is another methodology that is specifically devised forsoftware engineering of adaptive multi-agent systems.ADELFE is based on the Rational Unified Process (RUP)[15] and its objective is not to add another methodology butto work on some aspects not already considered by existingmethodologies, such as complex environment, dynamicsoftware adaptation.Similarly, to the research projects mentioned above, ourresearch presents a knowledge-based software engineeringframework for a special category of intelligent systems. Inour case, this category concerns IUIs that incorporate multi-criteria theories. The resulting framework of our research iscalled MBIUI. Like ADELFE, MBIUI is based on the RUP.RUP is an object-oriented process that advocates mul-tiple iterations of the software development process. It di-vides the development cycle in four consecutive phases: theinception, the elaboration, the construction and the transi-tion phase. Each phase is divided in four procedural steps,namely requirements capture, analysis and design, imple-mentation and testing. The phases are sequential in timebut the procedural steps are not.RUP is clearly documented and easily used due to itsclarity as has been pointed out in [11] where RUP has beencompared and contrasted to other software engineeringprocesses, such as Catalysis [16] and OPEN [17]. Addition-ally to its clarity, RUP is an object oriented process, thus itis appropriate for the development of graphical user inter-faces such as the one described in our research. Moreover,one important advantage of RUP is the highly iterative na-ture of the development process. For the above reasons,RUP has been selected as the basis for the MBIUI life-cycleframework. However, RUP does not provide specificguidelines about what sort of experiments or prototypes areneeded and when. Such information concerning IUIs thatare based on multi-criteria decision making, is given by theMBIUI life-cycle framework, which is described in this pa-per. 2.2 Intelligent User Interfaces Several researchers have dealt with the issue of designingand developing IUIs using theories from various researchareas. In addition to other theories, decision making theo-ries have also been used as reasoning methods in IUIs. Onecommon application area of decision making theories inIUIs is e-commerce. The theory that has been used mostextensively in this area is MAUT (e.g. [2], [4], [5], [18]). Allof the above mentioned projects have focused on adaptinga theory in order to improve the user interface by dynami-cally selecting the most appropriate e-commerce product tobe recommended to a user. However, there is a shortage ofreports on experimental studies that are needed throughoutthe life-cycle of such IUIs that base their reasoning on mul-ti-criteria theories.If a user interface incorporates intelligence, the complex-ity of the development of the system increases dramatically.This is even more the case when multi-criteria theories aremeant to be incorporated into the reasoning of the system.Interesting research on this subject has been conducted byBohnenberger et al. [3] who have presented the studies thatare needed for developing a decision-theoretic location-aware shopping guide. However, these studies mainly fo-cus on the evaluation and the possible improvements of the  3 system that incorporates decision theoretic planning andnot the whole life-cycle. Therefore it is our goal to presentthe life-cycle of an IUI that incorporates a decision-makingtheory and address the problems that are related to suchincorporation.As a test-bed for the corpus of our research we haveused a file manipulation environment like Microsoft Win-dows Exlporer. The aim of our research is to generate au-tomatic assistance in cases where users have made mistakeswith respect to their hypothesised intentions. Another helpsystem, which is similar to the domain of our research butdifferent in its basic rationale is Tip Wizard that has beenintroduced by Microsoft. Tip Wizard is quite well known tousers of Windows who remember the animated agent of thepaper-clip. However, this paper-clip was often criticised asannoying [19]. Tip Wizard’s main objective is to recom-mend new commands to users. This is done based on alter-native commands’ equivalence to the less efficient com-mand sequence that a user may be using in order to per-form a task. In contrast, in our research, the objective of theintelligent help system is to intervene only when this isconsidered really necessary for helping the user achievehis/her goals without errors. In our approach we do notconsider giving comments to users on the actual way theychoose to accomplish their goals as long as this way is notleading to a failure. In this way, we keep the interventionsto a minimum to avoid annoyance of users. Therefore, inour approach, if the help system suspects that an actionwould not have the desired results for the user, it generatesalternative actions that would achieve these hypothesisedgoals. Interventions are made only in cases when the multi-criteria decision making theory has ranked very highly thealternative to be proposed to the user. Furthermore, anoth-er advantage of our approach, in contrast to the oneadopted by Microsoft, is that our system takes into accountinformation about the user’s goals, usual errors and mis-conceptions and, therefore, makes interaction adaptive toeach individual user. An example of the IUI’s operation ispresented below:A user in his attempt to organize his file store, he movesthe contents of two folders into a third one, named “Pro-grams”. Then the user accidentally selects the folder “Pro-grams” and issues the command delete. However, this ac-tion seems contradictory to the user’s goals as the particularfolder has just acquired new contents that may be useful forthe particular user. Therefore, the system generates alterna-tive actions and applies a decision making model in orderto select the one that seems more likely to have been in-tended by the user. As a result, the system proposes to theuser to delete another folder that has a similar name: “Pro-gram” and is empty. The folders “Programs” and “Pro-gram” have similar names and are neighbouring in thegraphical representation of the file store, thus the user mayhave mistakenly selected “Programs” instead of “Pro-gram”. In the reasoning of the system, a factor that was alsotaken into consideration for the selection of the commandto be proposed to the user was the fact that from the historyof the user’s actions the user had been characterised asprone to accidental slips. The actual selection of the “best”alternative command to be proposed to the user is based ona multi-criteria theory.Our research described in the present paper is based onprevious research of ours on IUIs [20], [21], [22]. In particu-lar, in [21] we argued that experimental studies wereneeded for the life-cycle of an IUI irrespective of the incor-poration of a multi-criteria theory. In that paper we de-scribed the experimental studies that were needed for thedevelopment of an IUI prototype system, which was calledIFM. IFM’s reasoning was primarily based on a cognitivetheory and did not incorporate any decision making theoryat all [20]. As a major improvement of our research thecognitive theory was combined with the multi-criteria deci-sion making theory, which is called SAW. The combinationof SAW with the cognitive theory is described in [22].However, the incorporation of a decision making theoryinto IFM revealed a greater role that decision making theo-ries can play into the reasoning of IUIs in terms of the waytheir specifications are formed, their functionality andmaintenance. At the same time, important questions wereraised: 1. What sort of experiments is needed for the effi-cient incorporation of a multi-criteria theory into an IUI. 2.How these experiments should be set up. 3. How these ex-periments differ depending on the particular multi-criteriatheory that is used. These questions are very crucial for theincorporation of a multi-criteria theory into an IUI. This isdue to the fact that this incorporation is a process that in-volves many stages and experimental studies that are dedi-cated to the purpose of incorporating a multi-criteriatheory. For example, the application of a decision makingtheory in an IUI requires conducting experiments so thatdecision making information may be acquired from thehuman experts. Therefore, a designer has to address issueslike setting up the experiments. Such issues, once deter-mined, can be re-used in other similar systems.The above questions motivated extensive further re-search that led to a brand new version of the IUI that is nowcalled MBIFM (Multicriteria Based Intelligent File Manipu-lator) and is based extensively on the decision makingtheory which is called TOPSIS [7]. The multi-criteria theoryhas been used for the whole reasoning of the system andthus the cognitive theory is not used any more. The life-cycle framework that was devised for the development ofMBIFM is called MBIUI and is described in the present pa-per. Similarly to [21], MBIUI is based on an adaptation ofRUP, which is an object-oriented software life-cycle model.However, unlike [21], MBIUI is dedicated to a software life-cycle model for the development of IUIs that incorporate adecision making theory and the main emphasis of the re-search described in the present paper has been put on theadaptation of decision making theories in IUIs. 2.3 Decision Making Theories According to Triantaphyllou & Mann [23] there are threesteps in utilising a decision making technique that involvesnumerical analysis of alternatives: 1) Determining the rele-vant attributes and alternatives 2) Attaching numericalmeasures to the relative importance of the attributes and tothe impacts of the alternatives on these attributes 3)Processing the numerical values to determine a ranking ofeach alternative.  4 The determination of the relevant attributes and their rela-tive importance is made at the early stages of the softwarelife-cycle and is performed by the developer or is based onan empirical study which may involve experts in the do-main. However, decision making techniques mainly focuson step 3. There are many decision making techniques andthey have similarities and dissimilarities. In the followingparagraphs we present shortly and discuss issues aboutfour decision making techniques, namely SAW, MAUT,DEA and TOPSIS.In typical decision making methods such as the SAW [7],[8], the alternative actions are ranked by the values of amulti-attribute function that is calculated for each alterna-tive as a linear combination of the values of the n attributes.The multi-attribute function used in SAW is also used inMAUT. However, the main difference of the particulartheory with SAW is that the two theories use different ex-periments for the calculation of the weights of the criteria.SAW and MAUT presuppose that the weights of the criteriaare calculated in the early phases of the theory’s applicationand do not change over time. However, the weights of thecriteria are calculated dynamically in other decision makingtheories such as the Data Envelopment Analysis (DEA) [10].1999). DEA is a non-parametric linear programming ap-proach to evaluate the relative efficiency of decision mak-ing units (DMUs) that use multiple inputs to produce mul-tiple outputs. Unlike SAW, MAUT and DEA, TOPSIS calcu-lates the relative Euclidean distance of the alternative froma fictitious ideal alternative. The alternative closest to thatideal alternative and furthest from the negative-ideal alter-native is chosen best. More specifically, the steps that areneeded in order to implement the technique are: 1) Scale thevalues of the n attributes to make them comparable 2) CalculateWeighted Ratings 3) Determine Positive-Ideal and Negative-IdealSolutions 4) Calculate the separation measure from the positive-ideal and negative-ideal alternative 5) Calculate Similarity Index-es . 3 MBIUI   L IFE -C YCLE F RAMEWORK MBIUI life-cycle framework is based on RUP. As alreadymentioned, RUP gives a framework of a software life-cyclethat is based on iterations. However, RUP does neither spe-cify what sort of requirements analysis has to take place norwhat kind of prototype has to be produced during eachphase or procedural step. Such specifications are providedby our MBIUI framework concerning IUIs that are based onmulti-criteria theories.The MBIUI framework is illustrated in Figure 1. In thisfigure, we have maintained the phases and proceduralsteps of RUP. Based on this, we have specified what kind ofprototype has to be constructed in each iteration and whatkind of experiment has to be conducted. Therefore, Figure 1represents our solution to this problem.According to MBIUI framework, during the inceptionphase, the requirements capture is conducted. During re-quirements capture, a non-intelligent version of the userinterface has to be evaluated. The usability evaluation ofnon intelligent version of the user interface has to be con-ducted in order to identify the usability problems of suchuser interfaces. The usability problems of non-intelligentuser interfaces that can be identified can serve as a basis forthe requirements specification of the intelligent version.Furthermore, during the requirements capture, a prototypeof the IUI should be developed. This includes the specifica-tion of the way that the generation of intelligent advice tobe proposed to the user takes place in the system. However,the first prototype cannot include a user model or the adap-tation of a multi criteria decision making theory, which re-quires further experiments. At this point the multi-criteriadecision making theory that seems most promising for theparticular application has to be selected. This decision maybe revised in the procedural step of requirements capture inthe phase of construction.According to MBIUI, in the inception phase, duringanalysis, two different experiments are conducted in orderto select the criteria that are used in the reasoning processof the human advisors as well as their weights of impor-tance. The experiments should be carefully designed; sincethe kind of participants as well as the methods selectedcould eventually affect the whole design of the IUI.In MBIUI life-cycle framework, the two experiments in-volve human experts in the domain being reviewed. Theseexperts should comment on the protocols collected duringthe requirements capture of the inception phase. In particu-lar, the human experts should define the criteria that theywould use if they had to give advice to the users of the pro-tocols. The criteria that are proposed by the majority of thehuman experts are selected. When the final set of criteria isformed, another experiment is conducted in which the hu-man experts that participated in the first experiment areasked about the weight of importance of each criterion intheir reasoning process. The setting of the second experi-ment depends on which decision making theory has beenselected.The information collected during the two experiments ofthe empirical study is further used during the design phaseof the system, where the decision making theory that hasbeen selected is applied to the user interface. More specifi-cally, in the elaboration phase, during design, the usermodelling component of the system is designed and thedecision making model is adapted for the purposes of theparticular domain. Kass and Finin [24] define the usermodel as the knowledge source of a system that containshypotheses concerning the user that may be important interms of the interactive behaviour of the system.In the elaboration phase, during implementation, the us-er modelling component of the system as well as the basicdecision making mechanisms are developed. As a result anew version of the IUI is developed which incorporatesfully the multi criteria decision making theory.In the construction phase, during testing, the IUI that in-corporates the multi-criteria decision making theory is eva-luated. The evaluation of IUIs is very important for theiraccuracy, efficiency and usefulness. Indeed, as Mc Tear [25]points out, the relationship between theory and practice isparticularly important in Intelligent Interface Technologyas the ultimate proof of concept that the interface actuallyworks and that it is acceptable to users. Similarly, Chin [26]points out that empirical evaluations are needed to deter-  5 mine which users are helped or hindered by user-adaptedinteraction in user modelling systems. He adds that the keyto good empirical evaluation is the proper design and ex-ecution of the experiments. However, he notes that empiri-cal evaluations are not so common in the user modellingliterature.In MBIUI, evaluation is considered important for tworeasons: 1) the effectiveness of the particular decision mak-ing theory that has been used has to be evaluated 2) theeffectiveness of the IUI in general has to be evaluated. Incase the version of the IUI that incorporates a particulardecision making theory does not render satisfactory evalua-tion results with respect to real users and human experts,then the designers have to return to requirements capture,select an alternative decision making model and a new ite-ration of the life cycle takes place. In transition phase, dur-ing testing, the decision making model that has been finallyselected is evaluated and possible refinements of that mod-el may take place, if this is considered necessary. 4 R EQUIREMENTS C APTURE   In the inception phase, during the procedural step of re-quirements capture a usability evaluation of a non-intelligent version of the user interface should be conductedso that the usability problems that the IUI has to addressmay be revealed.In the case of MBIFM, a usability evaluation of a stan-dard file manipulation program (Windows 98/NT Explor-er) was conducted. This evaluation aimed at identifyingusability problems of standard file manipulation programsso that these problems were addressed in the design ofMBIFM. For this reason we conducted an experiment,which involved both users and human advisors. One of themain aims of the empirical study was to categorise as manyusers’ plans as possible and to identify the most frequenterrors that expert and novice users may make while inte-racting with a standard explorer. In this way, we couldidentify limitations of standard file manipulation programs Fig. 1. The RUP as adapted for the MBIUI life-cycle framework
Related Documents
View more...
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!