Unsupervised Design of Artificial Neural Networks via Multi-Dimensional Particle Swarm Optimization

In this paper, we present a novel and efficient approach for automatic design of Artificial Neural Networks (ANNs) by evolving to the optimal network configuration(s) within an architecture space. The evolution technique, the so-called multidimensional Particle Swarm Optimization (AID PSO) re-forms the native structure of PSO particles in such a way that they can make inter-dimensional passes with a dedicated dimensional PSO process. So in a multidimensional search space where the optimum dimension is unknown, swarm particles can seek for both positional and dimensional optima. This eventually removes the necessity of setting a fixed dimension a priori, which is a common drawback for the family of swarm optimizers. With the proper encoding of the network configurations and parameters into particles, AM PSO can then seek for positional optimum in the error space and dimensional optimum in the architecture space. The optimum dimension converged at the end of a AID PSO process corresponds to a unique ANN configuration where the network parameters (connections, weights and biases) can then be resolved from the positional optimum reached on that dimension. The efficiency and performance of the proposed technique is demonstrated over one of the hardest synthetic problems. The experimental results show that AM PSO evolves to optimum or near-optimum networks in general.