Computational Optimization in Engineering - Paradigms and Applications. However, the simulation is often a legacy code or commercial software whose inner workings are inaccessible to the user, and so there is no analytic expression that defines how candidate designs are mapped to objective values. Each simulation run is computationally expensive , that is, it requires a lengthy run time, and this severely restricts the number of candidate designs that can be evaluated.

However, the integration of metamodels and ensembles into the optimization search introduces several challenges: Locating a good solution requires effectively searching the metamodel, which can have a complicated landscape with multiple local solutions, and hence can be a difficult task. Since function evaluations are expensive, only a small number of evaluated vectors will be available and hence the metamodel will be inaccurate.

In severe cases, the optimization search can converge to a false optimum, namely, which was artificially created by the metamodel's inaccurate approximation of the true expensive function. In an attempt to address this issue, ensembles employ several metamodels concurrently and aggregate their individual predictions into a single one [ 4 , 5 ]. However, the effectiveness of ensembles depends on their topology, namely, which metamodels they incorporate, but the optimal topology is again typically unknown a priori , and may be impractical to identify by numerical experiments due to the high cost of each simulation run.

To address the issue of inaccurate metamodel predictions, the proposed algorithm operates within a trust region TR approach that manages the metamodel and ensures convergence to a valid optimum. Finally, to further improve the prediction accuracy the proposed algorithm uses ensembles and selects the most suitable topology during the search.

An extensive performance analysis based on both mathematical test functions and an engineering problem of airfoil shape optimization shows the effectiveness of the proposed algorithm. The remainder of this chapter is organized as follows: Section 2 provides the background information, Section 3 describes the proposed algorithm, then Section 4 provides an extensive performance evaluation and discussion, and finally Section 5 concludes this chapter. The latter was inspired by the physics of the annealing process in metals: initially a metal has a high temperature and so particles have a high probability of moving to a higher energy state.

As the metal cools in the annealing process, particles are more likely to move to a lower energy level than to a higher one. The process is completed once the system has reached the lowest possible energy level, typically its temperature of equilibrium with the environment. In the realm of global optimization, these mechanics have been translated into a heuristic search, which starts with an initial vector, namely, a candidate solution.

During the search, the current vector is perturbed so that new vectors in its vicinity are obtained. These vectors are evaluated and replace the original vector if: a they are better, or b they are worse and with probability p , which is analogous to the energy state changes. As p decreases, the search is transformed from being explorative to being more localized. Two main parameters of the SA algorithm are the annealing schedule , namely, the duration of the search process, which is determined by the manner that the temperature is decreased, and the selection probability function, which defines the dynamic threshold for accepting a worse solution.

Algorithm 1 gives a pseudocode of a baseline SA algorithm, while Section 3 gives the specific parameter settings of the SA implementation used in this study. The underlying mechanism of the SA algorithm was originally proposed by Metropolis et al. Since then the algorithm has been widely used in the literature, and some recent examples include [ 9 ] in finance, [ 10 ] in machine learning, [ 11 ] in chemical engineering and [ 12 ] in production line machine scheduling.

## How Simulated Annealing Works - MATLAB & Simulink - MathWorks India

Computationally expensive optimization problems are common across engineering and science. As mentioned in Section 1, metamodels are often used in such settings to alleviate the high computational cost of the simulation runs [ 2 , 3 ]. However, the integration of metamodels into the search introduces two main challenges: Prediction uncertainty: Due to the high cost of the simulation runs only a small number of designs can be evaluated, which in turn degrades the prediction accuracy of the metamodel and leads to optimization with uncertainty regarding the validity of the predicted objective values [ 13 ].

To address this, the metamodel needs to be updated during the search to ensure that its accuracy is sufficient to drive the search to a correct final result. To accomplish this, the proposed algorithm is structured based on the TR approach [ 14 ]. In this way, the algorithm performs a sequence of trial steps that are constrained to the TR, namely, the region where the metamodel is assumed to be accurate.

Based on the success of the trial step, namely, if a new optimum has been found, the TR and the set of vectors are updated. Section 3 presents a detailed description of the TR approach implemented in this study. Metamodel suitability: Various metamodel variants have been proposed, but the optimal variant is problem dependant and is typically unknown a priori [ 15 ]. To address this, ensembles employ multiple metamodels concurrently and combine their predictions into a single one to obtain a more accurate prediction [ 4 , 5 , 16 ].

The ensemble topology , namely, which metamodel variants are incorporated, is typically determined a priori and is unchanged during the search. However, the topology directly affects the prediction accuracy, and hence an unsuitable topology can degrade the search effectiveness. The same testing and training samples were used with all the topologies sized 30 and 20 vectors, respectively , such that each ensemble was trained with training sample and its prediction accuracy was tested on the testing sample.

The prediction accuracy was measured both with the root mean squared error RMSE ,. The overall prediction is the aggregation of the individual predictions. Addressing this issue, the algorithm proposed in this study selects the most suitable ensemble topology during the search, as explained in the following section. This section describes the algorithm proposed in this study, which is designed to address the issues described in Sections 1 and 2.

The proposed algorithm operates in five main steps, as follows:. Step 1.

### More Information

Initialization : The algorithm begins by generating an initial sample of vectors based on the optimal Latin hypercube design OLHD method [ 17 ]. After generating the sample, the vectors are evaluated with the true expensive function. Step 2. In turn, each candidate metamodel variant is trained with the training set and is tested on the testing set. The prediction accuracy is measured with the root mean squared error RMSE , which is calculated as.

The general structure of the algorithm requires an "energy function" the function to be optimized or some transformation of it , and an "annealing schedule", which tells what the initial "temperature" is and when and by how much it "cools". With RATS, the probability of moving from the current energy level e to a proposed energy level e' when at temperature T is.

It's possible to make a "bad" move to a higher energy level and it's also possible to not make a "good" move. However, the probability of either of those goes down as T decreases. Some descriptions of SA have this set to always take a good move. That's common to all uses of simulated annealing. What's specific to an application is the generation of possible moves proposals. For the traveling salesman problem, what's been found to work is a random choice of path reversals and path breaks.

- Recommended for you!
- [] A tensor network annealing algorithm for two-dimensional thermal states.
- Bibliographic Information.
- Words and Their Meanings.

However, we're interested in applying this to functions defined in continuous parameter spaces, not integers. Corana et. Each parameter maintains its own range which they call v i. This is because this schedule is very slow. For the microcanonical annealing, the total energy is equal to 70, Table 1 shows the parameters of the different cooling schedules used in this study. These parameters have proven to give good results. Numerical results obtained in this study for the left disparity maps are defined in the Table 2. The numerical results of the right disparity maps are nearly the same and do not necessitate to be shown separately.

The analysis of the results described in Table 2 show that the cooling schedule arithmetic-geometric and WCS have produced a high per-centage of correct matches with a small advantage for arithmetic-geometric cooling schedule.

## Journal of Intelligent Systems

We note As seen in Table 2 , logarithmic cooling schedule and WCS algorithm do not have a time record because these variants are time consuming. These variants are stopping after blocking. In Figure 2 , we show the visual of stereo matching for each cooling schedule, as we observe, the simulated annealing produced some amount of occlusions. Disparity maps computed with different cooling schedules and variants. Figure 2 shows the disparity maps obtained by each cooling schedule. For each cooling schedule, the left image represents the disparity map of the left-right direction and the right image represents the disparity map of the right-left direction.

We observe also that the geometrical cooling schedule and microcanonical annealing produce good results in terms of computation time and produce few errors compared to other alternatives. The arithmetic-geometric cooling schedule de-creases rapidly. Although it gave good result, this scheme could have trapped the algorithm in a local minimum. We conducted our experiments with different initializations: disparity map from correlation, disparity map with 0 values, and initialization with synthesized solution.

- A Farewell to Arms.
- Average-Cost Control of Stochastic Manufacturing Systems.
- (Simulated) Annealing algorithm.
- Strong and Weak Topology Probed by Surface Science: Topological Insulator Properties of Phase Change Alloys and Heavy Metal Graphene?

We also made sure the random number generator is always initially fed with a different seed point. For all these cases, we obtained a roughly similar energy value solution. The disparity maps are also almost identical, which confirms that our implementations achieve a strong minimum in finite time. Figure 3 shows the evolution of each term of the energy functional. We observe that during the first plateaus, some cooling schedules exploit the solution space by accepting the transition which increases the energy function when the temperature is very high.

Figure 3 illustrates the function values of each cooling schedule. Subfigure a shows the resemblance term, the second subfigure b represents the continuity term, the third subfigure c represents the occlusion term and the last subfigure d shows the global energy function. As seen in Figure 3 , we observe that the continuity and the resemblance are decreasing and converging to small values.

The global energy generated by geometric cooling schedule decreases rapidly in the first iterations. Thereafter, it decreases slowly. In this paper, we have studied and analyzed the perfermances of different cooling schedules used in conjonction with the simulated annealing and other variants without cooling schedule. The experimentations were conducted in the context of stereo matching. The microcanonical annealing and the geometrical cooling schedule have proven their perfermances in matching with an advantage for microcanonical annealing which produces a small number of occlusions.

The logarithmic schedule is very slow and it is unrealizable in practice. The results obtained by WCS show that this approach is also very slow. Last but not least, regardless of the initial state, our implementation of the S. It is important to note that the disparity maps are almost identical and the final value of the energy functional is almost the same. The work presented in this paper was financially supported by a project grant from University of Taif project number The authors are grateful to the institution for supporting their work.

Barnard, S. T Stochastic stereo matching over scale. International Journal of Computer Vision , Vol. Benameur, W Computing the initial temperature of simulated annealing. Computational Optimization and Applications , Vol. Geman, S. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. Hajek, B Cooling schedules for optimal annealing.

Mathematics of Operations Research , Vol. Huq, S. Stereo-based 3D face modeling using annealing in local energy minimization. ICIAP , pp. Jmaa, A. A new approach for hand gestures recognition based on depth map captured by rgb-d camera. Kirkpatrick, S. Optimization by simulated annealing. Science , Vol. Lecchini-Visintini, A. Simulated annealing: Rigorous finite-time guarantees for optimization on continuous domains.

Neural Information Processing Systems. Liu, L. Depth reconstruction from sparse samples: Representation, algorithm, and sampling. Malathi, T. Estimation of disparity map of stereo image pairs using spatial domain local gabor wavelet. Metropolis, N. Equation of state calculations by fast computing machines.