A Cluster-based Co-evolutionary Optimization Method for 4 Bilevel Multi-objective Optimization

14 The research motivation of multi-objective bilevel optimization mainly stems from the need to solve practical problems 15 and improve decision-making efficiency. On the one hand, bilevel optimization helps to solve the complexity and 16 uncertainty in real life, thereby improving decision-making efficiency and robustness. On the other hand, by promoting 17 the development and application of AI technology, bilevel optimization also provides support for sustainable devel-18 opment. Although the application of bilevel optimization has proven to be beneficial in addressing various real-life 19 problems. However, recent studies indicate that achieving both high speed and high-quality optimization through exist-20 ing algorithms remains challenging. This difficulty arises due to the NP-hard nature of the bilevel optimization problem. 21 The nested structure method, commonly used to tackle this problem, involves each upper level solution independently 22 performing the lower level optimization task. This approach significantly increases the number of evaluations for 23 the lower level. To address this issue, our proposed method leverages the similarity in lower level optimization to 24 group upper level solutions, enabling co-evolution of lower level solutions within the same group. Consequently, this 25 approach substantially reduces the number of evaluations required for lower level solutions. Additionally, our method 26 pairs parents and offspring, the optimized lower level solutions of the parents are utilized to optimize the lower level 27 solutions of the offspring. This approach accelerates the optimization process for the lower level. To validate the 28 effectiveness of our algorithm, we have applied it to a suite of test problems, demonstrating satisfactory performance.


INTRODUCTION
Today's society faces two major challenges: sustainable development and optimal use of resources.Green computing aims to reduce the environmental impact of computers and information technology, including reducing energy consumption and reducing e-waste [1] .Bilevel multi-objective optimisation is an optimisation framework that considers multiple objective functions at the same time to find a good balance.By incorporating green computing principles, such as minimizing energy consumption and reducing environmental impact, as additional objectives or constraints within the optimization framework, we can effectively prioritize sustainability alongside other performance metrics.The bilevel optimization problem (BLOP) is a nested structural problem, where a lower level optimization problem is embedded within the upper level optimization problem [2] .Bilevel optimization has its roots in game theory [3] .It has gained increased attention due to its potential usage in various fields, such as economic management, resource allocation, energy sustainability, transportation planning, machine learning and medical engineering [4][5][6][7][8][9][10] .In some cases, both the upper level and lower level optimization problems may contain multiple conflicting objective functions, leading to multi-objective bilevel optimization problems (MBOPs).Without loss of generality, a MBOP can be expressed as follows: where f  and f  are the upper level and lower level objective functions, respectively; x  and x  refer to the upper level and lower level solutions, respectively;   and   indicate the number of upper level and lower level objective functions, respectively;    x  , x  and    x  , x  are the th upper level and lower level constraints, respectively; and   and   denote the number of upper level and lower level constraints, respectively.If and only if the upper level constraints are satisfied and x  is a Pareto-optimal solution to the lower level optimization problem with regard to the given x  , then a solution x = (x  , x  ) is a feasible solution of a BMOP.The goal of solving a BMOP is to find a set of widely distributed feasible solutions with respect to the upper level optimization problem.Due to the nested structure, traditional optimization methods usually have difficulty solving MBOPs effectively.Evolutionary algorithms (EAs)     have been used to solve MBOPs [11,12] because they do not depend on the mathematical characteristics of MBOPs.
The single-level reduction approach converts a MBOP into a single-level optimization problem and then applies EAs to solve it.For instance, Li et al. [13] used adaptive weighting sum scalarization and Karush-Kuhn-Tucker (KKT) conditions to transform a BMOP to a multi-objective optimization problem (MOP).Then they proposed an effective smoothing technique to cope with complementarity constraints.Li et al. [15] improved the algorithm presented in [13] .
They first used the KKT condition to transform a BMOP into a MOP involving complementarity constraints and then proposed a decomposition-based constrained multi-objective differential EA.Jia et al. [14] converted a MBOP into a MOP based on the primal and dual theory and used multi-objective metaheuristics and constraint processing techniques for optimization.The method of converting a BMOP into a single-level optimization problem is very mature in the field of bilevel optimization, but these methods usually require strict mathematical assumptions, and these assumptions are usually difficult to satisfy in the real world.
The nested method solves the MBOP directly by performing the lower level optimization independently for each upper level solution.For example, Deb et al. [22] used elite nondominated ranking GA or NSGA-II for both upper and lower level optimization.This algorithm was subsequently upgraded by Sinha et al. [25] .Their improved version evaluates the upper level only once, which significantly reduces the number of evaluations of the algorithm.Second, it also allows the file members to participate in crossover, which improves the performance of the algorithm.Deb et al. [26] proposed a hybrid EA combined with a local search strategy for optimization.Cai et al. [23] proposed a divide-and-conquer strategy, in which all variables (both upper and lower levels) are divided into multiple mutually exclusive groups and optimized separately.Although these methods are successful, due to the structural characteristics of bilevel optimization, they cannot afford the large number of evaluations required to be consumed by the lower level evaluation when the number of upper level solutions is large.In order to solve the more complex MBOP, many new algorithms are produced, and a genetic algorithm is adopted at both levels.For example, both the upper and lower levels use Particle swarm optimization, and both upper and lower levels use differential evolution (DE).
The surrogate-model-based methods use a surrogate model to approximate the constraints and objective function at the lower level, with the aim of reducing the number of lower level fitness evaluations (FEs).For example, Sinha et al. [20] proposed an approximation-set-mapping approach which used quadratic functions to address the lower level optimization problem.Sinha et al. [21] modeled the lower level decision variables using a value function named m-BLEAQ; they used a quadratic function to estimate unknown lower level decision variables.It is essential to note that the accuracy of the surrogate model has a significant impact on the performance of final solutions.
In the real world, we are often faced with a complex set of problems, which often require multiple goals to be considered at the same time, and each goal may have a mutually constrained relationship.In order to solve this kind of problem, the bilevel optimization algorithm provides an effective solution.The bilevel optimization algorithm decomposes the problem into two levels, the upper optimization part is responsible for generating a set of feasible solutions, and the lower optimization part further finds the lower optimal solutions that meets the constraints according to this set of feasible solutions.This strategy of decomposing the problem enables the bilevel optimization algorithm to comprehensively consider multiple aspects of the problem, so as to obtain more comprehensive and high-quality solutions.Most of the current MBLOPs are from the perspective of game theory, using the concept of objective function values and Pareto optimality.In contrast, Gu et al. [27] are the first to study the convergence characteristics of MBLOPs problems from the perspective of traditional optimization, and consider a minimum and maximum robust version of the multi-objective problem, weighing the optimality of different objectives, and ensuring that each objective obtains a single optimal solution, rather than generating multiple Pareto optimal solutions.Recently, Wang et al. [28] designed a lower level environment selection strategy and an upper-level solution regeneration strategy to improve the search efficiency of the algorithm.Inspired by the biological classification of species, Mezura-Montes et al. [29] proposed a novel evolutionary framework based on the concept of family.They adopt the concept of science in biology to promote the diversity of solutions.
However, the implementation and application of the bilevel optimization algorithm still face many challenges.First, the complexity of bilevel optimization problems tends to be high, making the solution process very difficult.Secondly, the existing bilevel optimization algorithms are often difficult to balance the relationship between solution speed and solution quality.In order to solve these problems, we need to research and develop new bilevel optimization algorithms to improve the speed and quality of solutions.
In bilevel optimization, the lower level optimization tasks are usually relatively similar for multiple close upper level solutions.Inspired by this, based on the above observations, we design a new algorithm called CCBMO which utilizes match and cluster.To our knowledge, this is the first time that bilevel optimization has been performed using the similarity of parent and child lower level optimization tasks.The main contributions of this article are summarized as follows: 1. Initially, in the upper level decision space, we match a parent to each offspring based on the Euclidean distance.
This makes it easier to select a partial solution from the parent's lower solution as the initial solution when performing a lower optimization for a child, greatly reducing the number of lower level optimizations required.
2. Then, the k-means is used to cluster the upper level offspring into   /2 groups.When solutions in the same group are being optimized at the next level, we use co-evolution.
3. Following this, environment selection is performed based on upper level objective values, and selected upper level solutions are chosen for iteration.
The remainder of this paper is structured as follows: Section II presents the proposed bilevel optimization algorithm.
Section III provides a comprehensive experimental study, analysing and discussing the results.Finally, Section IV summarizes the findings and the potential for further research.

General Framework of Proposed Algorithm
The framework of the CCBMO algorithm is established by Algorithm 1.Initially, a set of   upper level solutions Then X  is clustered into   /2 groups using the k-means, denoted as G 1 , . . ., G   /2 (line 9).K-means is a clustering method based on Euclidean distance, which aims to minimize the sum of the distances of N objects from the nearest center point, and to divide these objects into K groups according to the size of the K value, and to consider the individuals in the same group to be similar.The index of each solution in th group in X  is stored in G  .Following the execution of the k-means clustering, the solutions belonging to the same cluster exhibit proximity, suggesting similarities among the corresponding tasks of lower level optimization.Therefore, the lower level optimization is performed simultaneously on the solutions belonging to the same group based on Algorithm 2. After that, the optimized lower level solutions and their corresponding upper level solutions are subsequently stored in L  (lines 11-14).The parents in L  and offspring in L  are combined, denoted as L.Then,   distinct upper level solutions are chosen to replace the original solutions in X  , based on the nondominated sorting and crowding distance obtained from the upper level environmental selection.The selected upper level solutions and their corresponding lower level solutions are stored in L  .Finally, the nondominated solutions in L are selected and merged into X  to update it.
In order to select the better solution for each iteration, the selection operator of NSGA-II is applied to L to obtain X  , and X  is used to store   distinct upper level solutions.This selection is based on the upper level objective values and the upper level constraint violation.To be specific, the solutions in X  are initially divided into different nondominated sets using the constrained-domination principle.According to this principle, if we have two solutions, x 1 and x 2 , x 1 is considered better than x 2 if any of the following conditions are met: 1.  (x 1 ) = 0 and  (x 2 ) = 0, and x 1 Pareto dominates x 2 ; 2.  (x 1 ) = 0 and  (x 2 ) > 0; 3.  (x 1 ) > 0 and  (x 2 ) > 0, and  (x 1 ) <  (x 2 ).
Then, the best half of the upper level solutions are added to X  .

Lower Level Optimization
Algorithm 2 provides the process of lower level optimization.First, an empty set S is created (line 1).Then we use

Termination Criterion
Termination conditions for both upper and lower layers are based on their hypervolume (HV).The rate of convergence of HV for both the upper and lower levels is called the H-metric and is calculated as follows: where  max and  min denote the maximum and minimum values of the hypervolume in the output solutions, respectively, and ′  ′ , ′  ′ are used to distinguish the upper and lower levels respectively.We use the maximum objective value of the nondominated solution as the reference point to calculate the HV.
When the number of iterations exceeds , and  < , the algorithm terminates.We set the termination conditions

EXPERIMENTAL STUDIES
In this section, the experimental study for investigating the performance of the proposed algorithm is presented.
First, we introduce the test problems and parameter settings.Then we briefly describe the comparison algorithm.
Afterward, we introduce the performance metrics used to evaluate the algorithm's performance.Finally, we give the experimental results and analyze the results.

3.1.Test Problems
Two benchmark test sets, TP and DS [26] , were selected for the experimental studies.In the TP test suite, we chose two test problems, TP1 and TP2, and the DS test suite includes five test problems (denoted as DS1-DS5).
The detailed settings for the TP test set and DS test set are provided in Table 1.  and   represent the dimensions of the upper and lower level decision variables, respectively.The dimensionality of both upper and lower level decision variables is K=5 for DS1-DS3.For DS4-DS5, the dimensionality of the upper level decision variables is K=1, and lower level decision variables is K+L=10.The other parameters of the comparison algorithms remain the same as in their original papers.

3.2.Compared Algorithms and Parameter Settings
In order to evaluate the performance of the algorithms, two algorithms were chosen for comparison: BLMOCC [23] , MOBEA-DPL [30] .
1) BLMOCC: it is an algorithm that uses a knowledge-based variable decomposition strategy to solve a MBOP.The knowledge-based variable decomposition strategy is used throughout the optimization process, and the variables are divided into three groups according to the correlation between the variables in the two levels.Different optimization methods are used independently for different groups.
2) MOBEA-DPL: it is an algorithm developed on the framework of the nested bilevel multi-objective optimization algorithm.It uses a dual-population optimization strategy to improve the solution of the lower level optimization problem.The first group is used to store nondominated solutions in the lower level, and the second group is used to store upper level solutions that are not dominated by solutions in the first group.Additionally, to increase the effectiveness of the search, the offspring of the upper level solutions are selected from the neighborhood of the best solutions in the existing solutions.

Table 1. SE TTINGS OF THE TEST PROBLEMS
Test Problem

3.3.Performance Metrics
We limit the maximum FEs of the upper and lower levels to facilitate comparing the number of true evaluations of different algorithms.The maximum number of evaluations for the upper and lower levels is 50,000 and 1,000,000, respectively.  and   represent the number of evaluations in the upper and lower levels, respectively.In addition, we compare the sum of the actual number of evaluations in the upper and lower levels (i.e.,   +   ).
Without loss of generality, we consider using the inverted generational distance (IGD) [31] and hypervolume (HV) [32] to measure the performance of the algorithms.Both metrics can measure the diversity and convergence of the obtained solutions.A smaller IGD value means better algorithm performance, while the opposite is true for HV values, where a larger HV value means better algorithm performance.

3.4.Comparison between CCBMO and other algorithms on each index
In this case, we measure whether the algorithm can converge well with a small number of FEs.We recorded the average of FEs (including the upper level, lower level, and the sum of both levels), the IGD, and HV for all test problems.Specifically, each algorithm ran independently 21 times.The best results for each test problem are highlighted in gray.

Table 2. PERFORMANCE COMPARISON BETWEEN BLMOCC, MOBEA-DPL AND CCBMO REGARDING THE AVERAGE VALUES OF FEs ON TP AND DS.
Problem From Table 2, it can be seen that CCBMO has better results than BLMOCC and MOBEA-DPL in terms of lower level average FEs on TP1-TP2, DS1, and DS3-DS5, while slightly worse than BLMOCC in terms of upper level average FEs.This is because in lower level optimization, we introduce the idea of co-optimization, which optimizes similar lower level optimization problems at the same time, thus greatly reducing the number of optimizations and evaluations at the lower level level.However, as shown in Figure 3, due to the obvious advantage of CCBMO in the average FEs of the lower level, the sum of the average FEs of the upper and lower levels shows the best results, except that MOBEA-DPL outperformed CCBMO and BLMOCC on DS2.Obviously, this is all due to the improvement of the lower level optimization strategy.DS2 is used to assess the algorithm's capability to conduct extensive searches before converging on the frontier.For  DS3 involves discrete variables, and as the number of variables increases, the problem becomes increasingly challenging, making it difficult for traditional algorithms to address.MOBEA-DPL achieved the best IGD and HV on DS3.In contrast, CCBMO performed worse than MOBEA-DPL due to its utilization of a traditional nesting method for addressing this problem.BLMOCC had the worst IGD and HV.
Both DS4 and DS5 evaluate the algorithm's capability to search for the appropriate lower level frontier that corresponds to the upper level frontier.Additionally, DS4 necessitates identifying a specific point in the lower level solution that corresponds to the upper level solution.In the case of DS4, CCBMO achieved superior IGD and HV values, which are one and two orders of magnitude better than the corresponding IGD values of MOBEA-DPL and BLMOCC, respectively, and one order of magnitude better than the HV value of MOBEA-DPL.In the case of DS5, CCBMO had significant superiority over the comparison algorithms.In terms of IGD value, CCBMO

3.5.Comparison of CCBMO with its variants
To illustrate the effectiveness of the mechanism in CCBMO, we compared two variants of CCBMO, CCBMO-M and CCBMO-C, where CCBMO-M represents the variant that removes the k-means clustering in CCBMO, and CCBMO-C represents the variant that removes the matching between the upper level offspring and parents in CCBMO.Tables 5-7 present the results of CCBMO, CCBMO-M and CCBMO-C regarding the average number of FEs, IGD and HV over 21 runs on TP1-TP2 and DS1-DS5, respectively.
Table 5 gives a comparison of CCBMO and its variants in terms of average FEs.CCBMO consumed the least number of evaluations in solving TP1-TP2, DS2 and DS4.CCBMO-M performed slightly worse than CCBMO, consuming the least number of evaluations on solving DS1 and DS3.CCBMO-C only consumed the least number of evaluations when solving DS5.

CONCLUSION
Our work shows the existing bilevel multi-objective optimization algorithms ignore the similarity between the lower level optimization tasks when solving the problem, and thus consume a large number of FEs when performing the lower level optimization.Based on these findings, we have proposed a new optimization algorithm framework (called CCBMO), which utilizes the similarity of the lower level optimization tasks of the parents and offspring to accelerate the solution of the lower level optimization tasks of the offspring.CCBMO mainly includes two stages: a matching stage and a clustering stage.In the matching stage, upper level solutions of the parents and offspring are matched based on Euclidean distance.Each offspring selects the closest parent and inherits its corresponding lower level solutions.The clustering stage subsequently decomposes the offspring into   /2 groups using k-means.
Co-evolution is then performed on the lower level solutions within the same group.CCBMO was tested on seven problems by comparing it with two algorithms (i.e.BLMOCC and MOBEA-DPL).The results demonstrate the effectiveness of CCBMO.At present, the research on bilevel optimization is very limited, and more complex methods are usually used to solve the bilevel optimization problem.In this paper, the traditional genetic algorithm is used to solve the upper level and lower level optimization problems respectively, and the correlation between the upper level and lower level optimization problems is noted, so that the bilevel optimization problem can be solved by a traditional nested method.On the theoretical side, we will continue to study the bilevel optimization problem that solves the NP hard by simple genetic algorithm framework.In addition to this, we will further explore the application of multi-objective bilevel optimization in real life.
Although CCBMO performed well on most problems, we found that CCBMO did not converge well when solving test problems with a large number of decision variables.Therefore, further efforts are needed to develop an efficient selection strategy to address larger-scale BMOPs.

Figure 1 .
Figure 1.We provide an example of a block plot for matching and clustering in Figure1A-B.Left: match the solutions in X  with each solution in X  based on the Euclidean distance; Right: clustering of X  via kmeans;

Figure 2 .IGDFigure 3 .
Figure 2. Comparison of the average FEs statistical results of the three algorithms on TP1-TP2 and DS1-DS5.

Figure 4 .
Figure 4. Comparison of the statistical results of the average HV value of the three algorithms on TP1-TP2 and DS1-DS5.

Tables 6 and
Tables 7 compare the IGD and HV values of CCBMO and its variants (CCBMO-M and CCBMO-C), respectively.In terms of TP1 and DS2-DS5, CCBMO outperformed CCBMO-M and CCBMO-C in both IGD and HV values.For TP2, CCBMO obtained the best IGD value, but the HV value was slightly worse than CCBMO-M.CCBMO-M had the best IGD and HV values when solving DS1.On the other hand, CCBMO-C performed the worst, not obtaining the best IGD and HV values for any problem.Therefore, our experiments demonstrate the effectiveness of matching and clustering incorporated in CCBMO.
, x  ), . . .,     (x  , x  )}, s.t.   (x  , x  ) ≤ 0,  = 1, . . .,   , x  ∈ arg min f  = {   1 (x  , x  ), . . .,     (x  , x  )}, s.t.   (x  , x  ) ≤ 0,  = 1, . . .,   , (line 2).The solutions in L  are then ranked based on the upper level objective values and the degree of constraint violation, with the nondominated solutions being stored in X  (line 3).← Generate   upper level solutions randomly; 2 L  ← Perform lower level optimization for each solution in X  , and obtain the lower level solutions respectively; 3 X  ← Select nondominated solutions from L  ;  = x  1 , ..., x    ← Perform differential evolution on the solutions in X  ;  that is the closest to x   ; , G 2 , ..., G   /2 ← Cluster the solutions in X  into   /2 groups utilizing K-means;During the main loop, we execute the DE on X  , with the resulting offspring stored in X  = x x    is randomly generated.Subsequently, the lower level optimization is conducted for x   individually via Algorithm 2. The obtained lower level solutions and their corresponding upper level solutions are stored in L1 X  = x  1 ,x  2 , ..., x 4 while Not Termination do 5 X 6 for  = 1 :   do 7 x   ← Find a solution in X 10 L  = ∅ ; 11 for  = 1 :   /2 do 12 L  1 , ..., L  | G  | ← Perform lower level optimization for solutions in the G  group; 13 L  ← L  ∪ L  1 , ..., L  | G  | ; 14 end 15 L = L  ∪ L  ; 16 X  ← Choose   distinct upper level solutions from L based on the results of upper level environment selection; 17 L  = L  1 , L  2 , ..., L    ← Store the upper level solutions and their corresponding lower level solutions in X  ; 18 X  ← Select the nondominated solutions from L and merge them with X  ; 19 end 1 , ..., x    (line 5).For each solution in X  , the closest solution to x   is denoted as x   (lines 6-8).Next, we explain the several types of matches that can occur as a result.As shown in Figure 1, in the first case, x  1 is matched with its closest parent upper level solution, x  1 ; therefore, x  1 = x  1 .In the second case, x  2 and x  5 are both matched with x  2 , as they are the closest to it; therefore, x  2 = x  2 and x  5 = x  2 ; in the third case, no offspring are closest to the solutions of x  6 and x  7 .
to denote the index of the th element in G  .Since we match a solution x    corresponding to x   .Then we store the lower level solution corresponding to x   into .As shown in Fig. 2, the x  1 , ..., x  8 were divided into four groups using the k-means algorithm.For example, x  1 is in a separate group.Based on Fig. 1, x  1 is matched with x  1 , and the lower level solutions whose upper level solution is x  1 are stored in .x  2 and x  3 belong to the same group, but they are matched with solutions x  2 and x  3 , respectively.Therefore, the lower level solutions whose upper level solutions are x  2 and x  3 are placed in .

X
The index of the th element in G  ;  ← Find the lower level solutions from L  corresponding to the solution whose upper level solution is Select min {  /2, ||} solutions from S; 9 X  2 ← Generate   − min {  /2, ||} lower level solutions randomly; 10 X  ← X  1 ∪ X  2 . ← Perform differential evolution on the solutions in X  ;  ← A 1 ∪ A 2 ∪ ...A | G  | ; as   = 0.001;   = 0.001;   = 10, and   = 40 dollars.In addition, for upper level optimization, if   exceeds 40 Algorithm 2: Lower level optimization Input: G  , L  , X  ; Output: L  1 , ..., L  | G  | ; 1 S = ∅; 2 for  = 1 : |G  | do 3  ← 4 S 11 while Not Termination do 12 X 13 C ← X  ∪ X  ; 14 for  = 1 : |G  | do 15 A  ← Perform environmental selection for C; 1 , ..., L  | G  | ← Store A 1 , ..., A | G  | and their upper level solutions; and there are six consecutive generations where   remains constant, the algorithm terminates.Since we optimize the lower level solutions simultaneously for multiple upper level solutions, for all lower level optimization processes, if   exceeds 10 and there are five consecutive generations where the value of   remains unchanged, the lower level optimization terminates.

Table 4 . PERFORMANCE COMPARISON BETWEEN BLMOCC, MOBEA-DPL AND CCBMO REGARDING THE HV VALUES ON TP1-TP2 AND DS1-DS5.
outperformed both comparison algorithms by two orders of magnitude.The HV value of CCBMO is an order of magnitude better than that of MOBEA-DPL.Therefore, these experiments demonstrate the effectiveness of CCBMO in solving BMOPs.