Input/Output
Data-File: The data file is an ASCII-text file, consists of columns where the first column is descriptor/id of the data and followed by the data for each attributes/variables. The first row of the file should also contains the description or header/label for each columns.

Example:

id    x     y   
1     0     0
2     0     2
....
....

No. of attributes: Is the number of data attributes or variables in data.

Output: An output/report file contains the analysis of data (containing data summary, class centroids, number of iterations, summary etc.).
When the "Write separate file for each class" box is checked, the membership of each class is written in separate files, consists of:
~ group.txt files containing the membership at each group/class
summary.txt containing validity measures at each class (FPI, MPE and S)
See performance measurement.
Membership Output includes:
- membership for each data point in each class
- CI = Confusion Index, is a measure of the degree of class overlap in attribute space (see Burrough and McDonnell, 1998, Principles of Geographical Information Systems. Oxford University Press).
CI is calculated as CI = 1- (m[max] - m[max2])
- Statistics of membership for each class: mean, min & max
 

Algorithm
The Algorithm menu permits selection of the fuzzy algorithm to be used for subsequent analyses. The choices are: fuzzy k-means, fuzzy k-means with extragrades, and fuzzy k-means equal area.
Fuzzy k-means is based on the algorithm presented in Bezdek (1981). See Figure 1.
Fuzzy k-means with extragrades (deGruijter and McBratney, 1988) recognizes that some objects might not fit well in any of the groups that are formed and places these objects in an additional outlier group, the extragrades (see Figure 2). deGruijter and McBratney have assumed that there will be as many samples in the extragrade group as in a regular group.
Fuzzy k-means equal area partition the variables into classes using the fuzzy k-means algorithm, and making sure that the area of membership allocated to each class (group) is equal.
Fuzzy k-means with fuzzy covariance matrices (Gustafson & Kessel, 1978) calculates the Fuzzy covariance matrix for each number of classes and use it as distance norm.

Metric
The Metric menu permits the user to select the norm-inducing metric (Bezdek 1981) which is to be used by the fuzzy algorithm. The choices are: Euclidean, Diagonal or Mahalanobis.

The metric (or 'distance' measure) should be chosen with care. Euclidean should not be used where different attributes have widely varying average values and standard deviations, since large numbers in one attribute will prevail over smaller numbers in another. With the diagonal and Mahalanobis metrics, the input data are transformed before use. Choosing diagonal results in transformation of the data attributes into having equal variance. Choosing Mahalanobis results in transformation of the data set to one in which all attributes have zero mean and unit variance, and correlations between variables are taken into account.

The fuzzy algorithms use measures of distance only when placing data points in groups. They are insensitive to direction. A useful (although not entirely correct) analogy is to think of the algorithms as defining circles in two-dimensional space (or spheres in three-dimensional space, or hyperspheres in multi-dimensional space) to cover the data points. Consequently, the performance of the algorithms on data sets showing a markedly stratified structure (for example, data clustered in planes, or on lines) depends critically on the choice of metric. Separations measured by diagonal or Mahalanobis distance might be better than Euclidean distance, for they compensate for marked departures from the (hyper-) spherical shape. Moreover, better separations might be given when more attributes are used, because measurements of many properties are likely to provide more distinctions and thus greater dissociation.
For the discussion on the choice of distance-dependence metric see Odeh et al. (1992).
 

Number of Classes
The analysis will commence by separating the data into the minimum number of classes (groups) and will terminate after separating the maximum number of groups. It is best to conduct an analysis by examining a few groups at a time.
 

Numerical Parameters
The numerical parameters chosen will affect the operation of the fuzzy algorithm. They are grouped according to whether they affect all three models, or whether they are related only to those with extragrades. The following parameters apply to all models:  fuzzy exponent, random start, maximum iterations and convergence limit. The parameters alpha tolerance difference and alpha iterations apply to fuzzy k-means with extragrades.

Fuzzy exponent: The fuzzy exponent controls the degree of fuzziness of a classification. When it is made to (or close to) 1.0 a data point can belong to only one class, that is, the classification will be hard or non-fuzzy. If it exceeds 1.0, an individual may be given partial membership in more than one class, that is, the classification will be fuzzy. A fuzzy exponent of 1.3 works well with Euclidean distance but might need to be made smaller with Mahalanobis distance. If it is too small (i.e., too close to 1.0) the program will attempt divisions by zero and may crash. If too big, the individuals will be given equal memberships in all classes. When set near 1 (e.g., at 1.01) a hard classification is usually obtained. The value is not constrained at the upper end (see McBratney and Moore, 1985), and as it is increased the clustering becomes so fuzzy that no groups are distinguished. Different values should be explored.

Maximum iterations: The maximum Picard iterations performed. If a solution is not reached within the number of iterations shown, increase the number.

Stopping criterion: The program will judge that it has found a solution when successive iterations produce results that differ by less than the convergence limit.

Random start: The initial assignment of memberships to groups is performed randomly by the program when this option is selected.

Alpha: parameter alpha (a) determines the relative number of extragrades to intragrades. Setting a = 1 provide the fuzzy k-means (without extragrades), while a  = 0 will result in classifying all the data into extragrades. If  information on the value of a is known, a can be fixed to the value, otherwise iterations is performed to search for an optimal  value, which is set to 1/(no_class+1).

Alpha iterations: The extragrade algorithm requires that the average membership of the outlier (or extragrades) class be the same as the average membership of the other classes. The parameter that determines outlier group membership is called alpha  (a). Since the exact relationship between alpha and the outlier group membership is not known a priori, the algorithm must arrive at the correct value for alpha using an iterative procedure (Figure 2). The outer loop of the procedure serves to refine an initial estimate for alpha. With alpha fixed, the inner loop solves for the memberships in the k intragrade groups and single extragrade group. This solution is used to determine the average membership in the outlier group. If the average membership in the outlier group equals (to within a desired accuracy = difference tolerance) the average membership in all groups, the algorithm terminates. Otherwise, a new value for alpha is estimated (using earlier alpha values as a guide). The parameter alpha iterations is the maximum number of iterative alterations to alpha that the algorithm will attempt. User can see by looking at the iteration data output whether the algorithm is converging to a solution, or whether a larger number of iterations is required.

Difference tolerance: The convergence criterion for the alpha iterations.

Performance Measurement
The OFV (objective function value) decreases monotonically with increasing number of classes and increasing value of fuzzy exponent.
The value dJ/dphi is the derivative of OFV over the fuzzy exponent  phi  = -[(dJ/dphi) sqrt(class)] devised by McBratney and Moore (1985) to obtain an optimal value for phi. The best value of phi for a given class is at maximum of the curve when plotting  -[(dJ/dphi) sqrt(class)] vs phi. See Odeh et al. (1992) for more details.
FPI (Fuzziness Performance Index) estimates the degree of fuzziness generated by a specified number of classes.
MPE (Modified Partition Entropy) estimates the degree of disorganization created by a specified number of classes.
The optimum number of classes was established on the basis of minimising these two measures.
Fuzziness performance index (FPI) is defined as (Roubens, 1982)

where F is the partition coefficient

Modified Partition Entropy (MPE) (Roubens, 1982):

where H is the entropy function:

Separate distance (S) (Xie and Beni, 1991)

where J₂ is:

and d_min is the separation measurement, the minimum distance between the cluster centroids. For more definition and application of FPI (Fuzziness Performance Index), MPE (Modified Partition Entropy), seeMcBratney and Moore (1985) & Odeh et al. (1992) and for seperate distance (S) see Xie and Beni (1991). For recent paper on cluster validity see Pal & Bezdek (1995).
 

Jackknife
Jackknife analysis sample some percentage of the whole data and perform of fuzzy clustering on a class. This procedure is repeated n (e.g. 100)  times (n runs), and n series of classes' centroids are produced. Statistics can be calculated on the centroid. An output file containing the centroid at each run and the membership to the 'true' centroid (using whole dataset).
The minimum number of data to be left out of the analysis should be less than the square-root of the no. of data. (Efron, B. and Tibshirani, R.J. 1993. An Introduction to the Bootstrap.Chapman & Hall, NY.)
Under the 'Number of data box', enter the no. data & the % of data need to be sampled is automatically calculated.

Initialize membership (U) iter = 0 Repeat {Picard iteration}     iter = iter+1     Calculate class center (C)     Calculate distance of data to centroid ||X-C||     Update membership U'     U=U' Until ||U-U'|| <= tol_crit .or. iter = Max_iter

Figure 1. Fuzzy k-means algorithm
 

Define 0< a <1 Initialize a Calculate required mean extragrades membership:     u*req = 1 /(k+1) iter_alfa = 0 Repeat {Root Finding Loop}     Iter_alfa = Iter_alfa + 1     Brent's algorithm     Update a     Initialize membership (U)     iter = 0     Repeat {Fuzzy k-means iteration}         iter = iter+1         Calculate class center (C)         Calculate distance ||X-C||         Update membership U'         U=U'     Until ||U-U'|| <= tol_crit  .or. iter = Max_iter     Calculate mean extragrades membership u*     Calculate F(a) = |u*-u*req| Until F(a) <= dif_tol .or. iter_alfa = Max_alfaiter

Figure 2. Fuzzy k-means with extragrades algorithm.
 

FuzME
Download FuzME
ACPA
Running FuzMe