Parallel training considered harmful? Comparing series-parallel and parallel feedforward network training
Carregando...
Data
Autor(es)
Título da Revista
ISSN da Revista
Título de Volume
Editor
Universidade Federal de Minas Gerais
Descrição
Tipo
Artigo de periódico
Título alternativo
Primeiro orientador
Membros da banca
Resumo
Neural network models for dynamic systems can be trained either in parallel or in series-parallel configurations. Influenced by early arguments, several papers justify the choice of series-parallel rather than parallel configuration claiming it has a lower computational cost, better stability properties during training and provides more accurate results. Other published results, on the other hand, defend parallel training as being more robust and capable of yielding more accurate long-term predictions. The main contribution of this paper is to present a study comparing both methods under the same unified framework with special attention to three aspects: (i) robustness of the estimation in the presence of noise; (ii) computational cost; and, (iii) convergence. A unifying mathematical framework and simulation studies show situations where each training method provides superior validation results and suggest that parallel training is generally better in more realistic scenarios. An example using measured data seems to reinforce such a claim. Complexity analysis and numerical examples show that both methods have similar computational cost although series-parallel training is more amenable to parallelization. Some informal discussion about stability and convergence properties is presented and explored in the examples.
Abstract
Assunto
Redes neurais (Computação)
Palavras-chave
Neural network, Parallel training, Series-parallel training, System identification, Output error models
Citação
Departamento
Curso
Endereço externo
https://www.sciencedirect.com/science/article/pii/S0925231218309068