Application of Fuzzy Numbers to the Assessment of CBR Systems

Case-Based Reasoning (CBR) is the process of solving problems by properly adapting the solutions of similar (analogous) problems solved in the past. As an Artificial Intelligence’s method CBR has become recently very popular to information managers increasing the effectiveness and reducing the cost of various human activities by substantially automated processes, such as diagnosis, scheduling, design, etc. In this paper a combination is utilized of the Centre of Gravity defuzzification technique and of the Fuzzy Numbers for assessing the effectiveness of CBR systems. Our new fuzzy assessment approach is validated by comparing its outcomes in our applications with the corresponding outcomes of two traditional assessment methods, the calculation of the mean values and the GPA index.


INTRODUCTION
Roughly speaking, Case-Based Reasoning (CBR) is the process of solving problems by properly adapting the solutions of similar (analogous) problems solved in the past.CBR has currently grown to a generally cognitive theory for Problem Solving (PS) and Learning.
In this work we shall apply a combination of the Centre of Gravity (COG) defuzzification technique and of the Fuzzy Numbers (FNs) for assessing the effectiveness of CBR systems.Fuzzy Logic (FL), due to its nature of characterizing the ambiguous real situations with multiple values, offers more realistic resources than those of the Aristotle's bi-valued logic for assessment purposes.In particular, FNs play an important role to the FL mathematics, analogous to the role played by the ordinary numbers to classical mathematics.
The rest of the article is formulated as follows: In Section 2 a brief but comprehensive account is given of the CBR process.In Section 3 we present those elements from the theory of FNs and in particular of the Triangular (TFNs) and Trapezoidal (TpFNs) FNs which are necessary for the purposes of this work.In Section 4 examples are presented of applications of our new fuzzy assessment method to several types of CBR systems.Our method is validated by comparing its outcomes with the corresponding outcomes of two traditional assessment methods, the calculation of the mean values and of the Grade Point Average (GPA) *Address correspondence to this author at the Department of Mathematical Sciences, School of Technological Applications, Graduate Technological Educational Institute (T.E. I.) of Western Greece, Meg.Alexandrou 1, 26334 Patras, Greece; E-mail: mvosk@hol.grindex.Our last Section 5 is devoted to our general conclusions and a short discussion about future perspectives of further research on the subject.

CASE-BASED REASONING
One of the most popular general PS strategies is the strategy of the analogous problem: When the solver is not sure of the appropriate procedure to solve a given problem, a good hint would be to look for a similar problem solved in the past, and try to adapt the solution procedure of this problem for use with the new problem.The PS by analogy, usually referred as Analogical Reasoning (AR) or Analogy-Based Reasoning is a special case of the general class of transfer of knowledge, i.e. of the use of already existing knowledge to produce new knowledge [1].
The importance of AR in human thinking has been recognized years ago.In fact, there is a considerable number of studies developed and many experiments performed on individuals by mathematicians, psychologists and other scientists about the AR process ( [2], Section 2).However, it is the Case-Based Reasoning (CBR) approach for PS and learning that has got a lot of attention over the last few years, because as an intelligent-systems method enables information managers to increase efficiency and reduce cost of many human activities by substantially automating processes, such as diagnosis, scheduling and design ( [2], Section 3).Notice that the term AR is sometimes used as a synonymous of the typical CBR approach [3].However, it is often used also to characterize methods, that solve new problems based on past cases of different domains [4,5], while typical CBR methods focus on single-domain cases (a form of intra-domain analogy).
CBR is often used where experts find it hard to articulate their thought processes when solving problems.This is because knowledge acquisition for a classical knowledge-based system would be extremely difficult in such domains, and is likely to produce incomplete or inaccurate results.
When using CBR the need for knowledge acquisition can be limited to establishing how to characterize cases, i.e. the analogous problems' situations.A case-library can be a powerful corporate resource allowing everyone in an organization to tap in the corporate library, when handling a new problem.A CBR system, usually designed and functioning with the help of computers, allows the case-library to be developed incrementally, while its maintenance is relatively easy and can be carried out by domain experts.
There are two styles of CBR, the PS style and the interpretive style.The PS style can support a variety of tasks including planning, diagnosis and design, e.g. in Medicine [6], Industry [7] and Robotics [8].The interpretive style is useful for classification, evaluation or justification of a solution, argumentation and for the projection of effects of a decision.Lawyers and managers making strategic decisions use the interpretive style [9,10].
CBR has been formalized for purposes of computer and human reasoning as a four steps process, often referred as the "four R's".These steps involve: R 1 : Retrieve the most similar to the new problem past case.
R 2 : Reuse the information and knowledge of the retrieved case for the solution of the new problem.

R 3 :
Revise the proposed solution.

R 4 :
Retain the part of this experience likely to be useful for future problem solving.
The first three of the above steps are not linear, characterized by a backward -forward flow among them.A simplified flow -chart of the CBR process, adequate for the purposes of the present paper, is presented in Figure 1 below: More details about the CBR methodology, history and applications can be found in [2,11] and in the relevant references given in the previous papers.A detailed functional diagram illustrating the four steps of the CBR process is also available in [2] (Figure 1).As a general PS methodology intended to cover a wide range of real-world applications, CBR must face the challenge to deal with uncertain, incomplete and vague information.Correspondingly recent years have witnessed an increased interest in formalizing parts of the CBR methodology within frameworks of reasoning under uncertainty, and in building hybrid approaches by combining CBR with methods of uncertain and approximate reasoning.
In an earlier work [12] we have developed a mathematical framework for the CBR process by introducing a finite Markov Chain on it main steps, while in [13] we represented these steps as fuzzy subsets of a set of linguistic labels characterizing the degree of success of the CBR process in each step and we have utilized the corresponding CBR system's total possibilistic uncertainty for measuring its effectiveness.Also, in [14] we have applied the Trapezoidal Fuzzy Assessment Model (TpFAM), which is a recently developed variation of the Center of Gravity (COG) defuzzification technique [15], for assessing a CBR system's performance in each step of the CBR process.

Basic Definitions
FL is based on the notion of a Fuzzy Set (FS), introduced by Zadeh in 1965 [16] as follows:

Definition
A FS A on the universal set U (or a fuzzy subset of U) is a set of ordered pairs of the form Α = {(x, m Α (x)): x !U}, defined in terms of a membership function m Α : U ! [0,1] that assigns to each element of U a real value from the interval [0,1].
The value m Α (x) us called the membership degree of x in A. The greater is m Α (x), the better x satisfies the characteristic property of A. The definition of the membership function is not unique depending on the user's subjective data, which is usually based on statistical or empirical observations.However, a necessary condition for a fuzzy set to give a reliable description of the corresponding real situation is that its membership function's definition is compatible to the common logic.
For example, the probabilistic approach yields the natural language statement "there is an 85% chance that Mary is tall", while the fuzzy terminology corresponds to the expression "Mary's degree of membership within the set of tall people is 0.85".The semantic difference is significant: The first view supposes that Mary is or is not tall (still caught in the law of the Excluded Middle); it is just that we only have a 85% chance of knowing in which set she is in.In contrast, fuzzy terminology supposes that Mary is "more or less" tall, or some other term corresponding to the value of 0.85.
A crisp subset A of U can be considered as a fuzzy set in U with m Α (x) = 1, if x !A and m Α (x) = 0, if x ! A. In this way most properties and operations of crisp sets can be extended to corresponding properties and operations of fuzzy sets.
Note that many authors identify, for simplicity, a FS with its membership function, while others represent it as a symbolic sum (finite or an infinite series according to the cardinality of U), or as a symbolic integral, when U has the power of the continuous.
The process of reasoning with fuzzy rules involves:

•
Fuzzification of the problem's data by utilizing the suitable membership functions to define the required FSs.
• Application of FL operators on the defined FSs and combination of them to obtain the final result in the form of a unique FS.
• Defuzzification of the final FS to return to a crisp output value, in order to apply it on the real world situation for resolving the corresponding problem.
For general facts on fuzzy sets we refer to the book of Klir & Folger [17].
A FN is a special form of a FS on the set R of real numbers defined as follows:

Definition
A FN is a fuzzy set A on the set R of real numbers with membership function m A : R ! [0, 1], such that: a}, a in [0, 1], are closed real intervals, and As a counter example, Figure 2 represents the graph of a fuzzy set on R which is not convex.In fact, we observe that A 0.4 = [5, 8.5] ![11,13], i.e.A 0.4 is not a closed interval.The following statement defines a partial order on the set of all FNs:

Definition
Given the FNs A and B we write A ! B (or ! ) if, and . Two such FNs are called comparable, otherwise they are called non comparable.

Remark
One can define the four basic arithmetic operations on FNS in the following two, equivalent to each other, ways [18]: With the help of their a-cuts and the Representation-Decomposition Theorem of Ralesscou-Negoita ( [19], Theorem 2.1, p.16) for FS.In this way the fuzzy arithmetic is turned to the well known arithmetic of the closed real intervals. (ii) By applying the Zadeh's extension principle ([17], Section 1.4, p.20), which provides the means for any function f mapping the crisp set X to the crisp set Y to be generalized so that to map fuzzy subsets of X to fuzzy subsets of Y.
In practice the above two general methods of the fuzzy arithmetic, requiring laborious calculations, are rarely used in applications, where the utilization of simpler forms of FNs is preferred.
For general facts on FNs we refer to Chapter 3 of the book of Theodorou [20], which is written in Greek language, and also to the classical on the subject book of Kaufmann and Gupta [18].

Triangular Fuzzy Numbers
TFNs are the simplest form of  Therefore the analytical definition of a TFN is given as follows:

Definition
Let a, b and c be real numbers with a < b < c.Then the TFN (a, b, c) is a FN with membership function: The following two Propositions refer to basic properties of TFNs that we are going to use later in this paper:

Proposition
The x-cuts A x of a TFN A = (a, b, c), x ![0, 1], are calculated by the formula A Proof: The graph of the TFN (a, b, c) is the triangle ABC of Figure 3, with A (a, 0), B(b, 1) and C (c, 0).Then, the COG, say G, of ABC is the intersection point of its medians AN and BM.The proof of the Proposition is easily obtained by calculating the equations of AN and BM and by solving the linear system of these two equations.

Arithmetic Operations on TFNs
It can be shown [5] that the two general methods of defining arithmetic operations on FNs mentioned in Remark 3. In other words, the opposite of a TFN, as well as the sum and the difference of two TFNs are always TFNs.On the contrary, the product and the quotient of two TFNs, although they are FNs, they are not always TFNs, unless if a, b, c, a 1 , b 1 , c 1 are in R + [21].

Trapezoidal Fuzzy Numbers
Another simple form of FNs that are frequently used in applications are the TpFNs.Roughly speaking, a TpFN (a, b, c, d) with a, b, c, d in R states that "a certain real value lies in the interval [b, c]".Its membership function y=m(x) is constantly 0 outside the interval [a, d], while its graph in this interval is the union of three straight line segments forming a trapezoid with the X-axis (Figure 4), Therefore, the analytic definition of a TpFN is given as follows:

Definition
Let a < b < c< d be given real numbers.Then the TpFN (a, b, c, d) is the FN with membership function:

Remarks (i)
A TFN (a, b, d) can be considered as a special case of the TpFN (a, b, c, d) with b=c, i.e. the TpFNs are generalizations of TFNs.
(ii) It can be shown [18] that the arithmetic operations of addition and subtraction of TpFNs, as well as the corresponding two scalar operations can be performed in the same way with TFNs (see paragraph 3.2.4).However, the product and the quotient of two TpFNs, although they are FNs, in general they are not TpFNs, apart from some special cases [18].
The following definition is introduced to be used in Section 4 for assessing the performance of a CBR system with the help of TpFNs/TFNs:

Definition
Let A i , i = 1, 2,…, n be TpFNs/TFNs, where n is a non negative integer, n ! 2. Then we define the mean value of the A i 's to be the TpFN/TFN: A= 1 n We close this section with the following Proposition, which utilizes the COG technique for defuzzifying TpFNs:

Proposition
The coordinates (X, Y) of the COG of the graph of the TpFN (a, b, c, d) are calculated by the formulas Proof: We divide the trapezoid forming the graph of the TpFN (a, b, c, d) in three parts, two triangles and one rectangle (Figure 4).The coordinates of the three vertices of the triangle ABE are (a, 0), (b, 1) and (b, 0) respectively, therefore by Proposition 3.2.3 the COG of this triangle is the point C ). Similarly one finds that the COG of the triangle FCD is the point C 2 ).Also, it is easy to check that the COG of the rectangle BCFE, being the intersection of its diagonals, is the point It is well known then [22] that the coordinates of the COG of the trapezoid, being the resultant of the COGs C i (x i , y i ), i=1, 2, 3, are calculated by the formulas , where is the area of the trapezoid.
The proof of the Proposition is completed by replacing the above found values of S, S i , x i and y i , i = 1, 2, 3, in formulas (1) and by performing the corresponding calculations.

ASSESSING THE PERFORMANCE OF A CBR SYSTEM WITH THE HELP OF TFNs/TpFNs
In this Section, we provide applications in which the TFNs/TpFNs are used as tools for the assessment of the effectiveness of CBR systems.For this, ranking in a climax from 0 to 100 the effectiveness of a CBR system's past cases when used with new similar problems, we consider the following linguistic characterizations (or labels or grades or degrees) for their performance: A (85-100) = excellent, B (75-84) = very good, C (60-74) = good, D (50-59) = fair and F (0-49) = unsatisfactory 2 .This new fuzzy assessment approach is validated by comparing in our applications its outcomes with the corresponding outcomes of two classical assessment methods of the bi-valued logic, the calculation of the mean values and of the GPA index.

Application (GPA -TFNs)
Consider two CBR systems designed for help desk applications, with their libraries containing 105 and 90 past cases respectively.The designers of both systems have supplied them with the same mechanism (software) for assessing the degree of success of their The scores attached to the above linguistic labels are not standard, depending on the designer's personal criteria.For example, in a more strict evaluation one could consider A (90 -100), B (80 -89), C (70 -79), D (60 -69), F (0 -59), etc.
past cases when used with new similar problems.The outcomes of this mechanism are depicted in Table 1 for each of the three first steps of the CBR process: Here we shall use the GPA index and the TFNs as assessment methods: (i) GPA index: We recall that the Great Point Average (GPA) index is a weighted mean in which more importance is given to the higher scores by attaching greater coefficients (weights) to them [23].In other words, the GPA index focuses on the quality performance of a student group.Denote by y i , i = 1, 2, 3, 4, 5 the frequencies of the CBR system's cases whose performance is characterized by F, D, C, B and A respectively, then the GPA index is calculated by the formula GPA = y 2 + 2y 3 + 3y 4 + 4y 5 (2).
In case of the ideal performance (y 5 = 1) we have GPA = 4, while in the worst case (y 1 = 1) we have GPA = 0; therefore 0 !GPA ! 4. Consequently, values of GPA greater than 4 : 2 = 2 correspond to a more than satisfactory system's performance In our case, the data of Table 1 give the following frequencies: Replacing the values of frequencies from Table 2  index show that the first system demonstrated a better quality performance at steps R 1 and R 3 (Retrieve, Revise) , .while the second one demonstrated a better performance at R 2 (Reuse).Further, the two systems' performance was proved to be more than satisfactory in R 1 and less than satisfactory in the other two steps, being worse at R 3 .This was logically expected, since the success in each step depends on the success in the previous steps.Notice that the two systems' performance at the last step R 4 was not examined, since all past cases, even the unsuccessful ones, are retained in a system's library for possible use in future with related new problems; the unsuccessful ones to help for exploring possible reasons of failure to find a solution for the new problem.
Finally, the mean values of the GPA index for the two systems at the steps R 1 , R 2 and R 3 are approximately equal to 1.84 and 1.62 respectively, showing that the first system demonstrated a better overall performance.

(ii) Use of the TFNs:
We assign to each assessment grade a TFN (denoted for simplicity by the same letter) as follows: A= (85, 92.5, 100), B = (75, 79.5, 84), C = (60, 67, 74), D= (50, 54.5, 59) and F = (0, 24.5, 49).The middle entry of each of the above TFNs is equal to the mean value of the higher and lower scores attached to the corresponding degree, while the left and right entries are equal to the minimal and maximal score respectively of the corresponding degree.
Inspecting the data of Table 1 one finds that for the first system and in step R 1 we have 51 TFNs equal to C, 24 TFNs equal to B and 30 TFNs equal to A. Then, by Definition 3.3.3,the mean value of these TFNs, denoted for simplicity by the same letter R 1 , is equal to Observing the TFN R 1 one concludes that the performance of the first system at the step R 1 (retrieve) lies in the interval [70.57, 90.96], i.e. it could be characterized from good (C), to excellent (A).Further, since the mean entries of the TFNS A, B and C are taken to be equal to the mean values of the lower and higher scores assigned to each of them, the middle entry of R 1 , being the mean value of these means, gives a rough approximation only of the systems performance (obviously not equal to the mean system's performance, measured by the mean value of all scores assigned to the system's cases).Therefore, here the first system's performance could roughly been characterized by the score 82.89, i.e. as very good (B).
In the same way one calculates the mean values R 2 (36F + 30D + 39C) !(36.57, 48.86, 61.14), thus obtaining the analogous conclusions for the first system's performance at the steps R 2 and R 3 of the CBR process.Finally, the overall system's performance can be assessed by the mean value R = 1 3 (R 1 + R 2 + R 3 ) ! (52.71, 63.94, 73.75), which shows that it can be characterized from fair (D) to good (C).A similar argument gives for the second system the values R 1 = (62.5,68.25, 74), R 2 !(45.33, 55.17, 65) , R 3 = (33, .46.25, 59.5) and R ! (46.94, 56.56, 66.17), thus obtaining the analogous conclusions for its performance at each step of the CBR process and its overall performance.However, the above approximate assessment can be precisely checked by applying Proposition 3.2.2 for calculating the x-cuts of the TFNs R and Definition 3.1.3for comparing these two TFNs.In fact, the x-cuts of them are [52.71+ 11.23x, 73.75 + 9.81x] and [46.94 + 9.62x, 66.17 + 9.59x] for the first and the second system respectively, while obviously we have that 52.71 + 11.23x > 46.94 + 9.62x and 73.75 + 9.81x > 66.17 + 9.59x, for all x in [0, 1].This shows that (52.71, 63.94, 73.75) > (46.94, 56.56, 66.17), i.e. that the first system demonstrates a better performance.The outcomes of the assessment methods (GPA, and TFNs) applied in the present application are depicted in Table 3.
Our next application illustrates the usefulness of defuzzifying the TFNs in assessment processes:
Here we shall use the traditional calculation of the mean values and the TFNs as our assessment methods: (i) Mean values: Calculating the means of the scores for the two systems in the classical way one finds the values 76.006 and 75.09 respectively.This shows that the mean performance of both systems was very good (B), while the first system demonstrated a slightly better performance.
(ii) TFNs: Using the TFNs A, B, C, D and F of paragraph (iii) of Application 4.1 the above scores are depicted in the following Table 4: We observe that in Table 4 we have 170 TFNs for the first and 255 TFNs for the second system representing the performance of their past cases.For simplicity, let us denote the mean values of these TFNs by the letters S 1 and S 2 respectively.Then, performing straightforward calculations, one finds that

GPA index
The first system demonstrated a better overall performance and a better quality performance at steps R1 and R3, while the second system performed better at step R2

TFNs
The first system demonstrated a better performance.
contrary, we have that 83.47-11.73x!79.53-6.9x!3.94 !4.83x !0.82 !x, which is not true for all values of x.Therefore, according to Definition 3.1.3,the TFNs S 1 and S 2 are not comparable, which means that at this stage one can not decide which of the two systems demonstrates the better performance.
In order to overcome this difficulty we shall defuzzify the TFNs S 1 and S 2 .In fact, by Proposition 3.2.3, the COGs of the triangles forming the graphs of these Observe now that the GOGs of the graphs of S 1 and S 2 lie in a rectangle with sides of length 100 units on the X-axis (customer scores from 0 to 100) and one unit on the Y-axis (normal fuzzy sets).Therefore, the nearer the x-coordinate of the COG to 100, the better the corresponding system's performance.Thus, since X > X', the first system demonstrates a (slightly) better performance.
In concluding, the outcomes of the two assessment methods used in this application (means and TFNs) are compatible to each other.
In our next application we shall use the TpFNs also as assessment tools:

Application (TFNs -TpFNs)
Six different users of a CBR system ranked with scores from 0-100 the effectiveness of the following five past cases for solving new related problems: Here we shall evaluate the system's effectiveness with respect to the above cases using the TFNs and the TpFNs: (ii) TpFNs: We assign to each case C i , I = 1, 2, 3, 4, 5 a TpFN (denoted, for simplicity, with the same letter) as follows: C 1 = (0, 43, 52, 59), C 2 = (75, 81, 95, 100), C 3 = (75, 76, 98, 100), C 4 = (85, 86, 88, 100) and C 5 = (0, 35, 62, 74).Each of the above TpFNs characterizes the performance of the corresponding case in the form (a, b, c, d), where a is the lower bound of its performance with respect to the corresponding linguistic grades, b and c are the lower and higher scores respectively assigned to the case by the six system's users and d is the upper bound of its performance with respect to the linguistic grades.
Next, for assessing the overall system's performance with respect to the given five past cases, we calculate the mean value of the TpFNs C i , i =1, 2, 3, 4, 5, which is equal to the TpFN C = We note that in Applications 4.1 and 4.2 it was practically difficult to use the TpFNs as assessment tools due to the great number of the existing past cases.
In concluding, we observe that the outcomes of the use of the TFNs and of the TpFNs for the evaluation of the system's performance are compatible to each other.However, the information obtained is an approximation only, not giving an exact idea about the system's performance.For this, we need to defuzzify either the TFN M or the TpFN C. We shall do this for C by extending the present Application as follows:

Application (Defuzzification of TpFNs)
Reconsider Application 4.3 and assume that the same six users assessed also the performance of five similar cases of another CBR system designed for the same purposes with the first one (e.g.diagnosis of a disease).Assume further that the overall performance of these cases was assessed as in Application 4.3 (using TpFNs) and that the mean value of the corresponding TpFNs was found to be equal to C΄ = (47.8,65.3, 78.1, 85.9).How one can compare the overall performance of the two systems with respect to the given five past cases of each one of them?Solution: Applying Proposition 3.3.4we find that the x-coordinate of the COC of the trapezoid constituting the graph of the TpFN C is equal to " 68.84 .
In the same way we find that the x-coordinate X΄of the graph of C΄ is approximately equal to 68.13.Therefore, the performance of both systems can be characterized as good (C).
Further, using the same argument as in Application 4.2 for TFNs, we observe that the nearer the xcoordinate of the COG to 100, the better the corresponding system's performance.Thus, since X > X', the first system demonstrated a slightly better performance than the second one.

CONCLUSION
In the present paper we used the Triangular and Trapezoidal Fuzzy Numbers (TFNs and TpFNs) for the assessment of a CBR system's performance.Since two TFNs/TpFNs are not always comparable, the use of them as assessment tools was combined in certain applications with the COG defuzzification technique.The creditability of this new fuzzy assessment approach was validated by comparing the outcomes of our applications with the corresponding outcomes two traditional assessment methods (mean values, GPA index).
The use of the TFNs/TpFNs together with the COG technique has the potential of a general assessment method that could be used in a great variety of other machine (e.g. for decision-making, etc.) and human (e.g.student assessment, etc.) activities.This is indeed the main target of our future research on the subject.

Figure 1 :
Figure 1: A simplified flow-chart of the CBR process.

Figure 2 :
Figure 2: Example of a non convex fuzzy set on R. Since the x-cuts A x of a FN, say A, are closed real intervals, we can write A x = [ A l x , A r x ] for each x in [0, FNs.A TFN (a, b, c), with a, b, c in R actually means that "the value of b lies in the interval [a, c]".The membership function of (a, b, c) is zero outside the interval [a, c], while its graph in [a, c] consists of two straight line segments forming a triangle with the OX axis (Figure 3).

Figure 3 :
Figure 3: Graph and COG of the TFN (a, b, c).
x = [ A l x , A r x ] = [a + x(ba), c -x(c -b)] .Proof: Since A x = {y !R: m(y !x}, Definition 6 gives for the case of A l x that y ! a b ! a = x !y = a + x(b -a).Similarly for the case of A r x we have that c ! y c ! b = x !y = c -x(c -b).3.2.3.Proposition (Defuzzification of a TFN) The coordinates (X, Y) of the COG of the graph of the TFN (a, b, c) are calculated by the formulas X = a + b + c 3 , Y = 1 3 .

2 )
. Further, the areas of the two triangles are equal to S 1 = b ! a 2 and S 2 = d !c 2 respectively, while the area of the rectangle is equal to S 3 = c -b (in all cases the corresponding height is 1, since the TpFN (a, b, c, d) is a normal fuzzy set on R).

= 1 105(
18F + 18D+ 48C + 21B) !(51, 60.07, 69.14) and R 3 = 1 105 We consider again the TFNs A, B, C, D and F used in our two previous Applications.Observing the 5*6 = 30 in total scores of the given data one finds that in the present Application we have14 TFNs equal to A, 4 equal to B, 1 equal to C, 4 equal to D and 7 TFNs equal to F characterizing the five cases' performance.The mean value of the above TFNs is equal to M = 1 30 (14A + 4B + C + 4D + 7F) !(60.33, 68.98, 79.63).Therefore, the system's performance lies in the interval [60.33, 79.63], i.e. it can be characterized from good (C) to very good (B).

Table 3 : Outputs of the Assessment Methods Used in Application 4.1 METHOD OUTPUT
Therefore, the system's performance is lying in the interval [64.2, 79], i.e. it can be characterized from good (C) to very good (B).